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The  Use  of  Weather  Information 
IN  Aeronautical  Decision  Making:  II 


I.  INTRODUCTION 

This  report  describes  research  which  assessed  worth 
functions  that  a  nationwide  sample  of  pilots  assign  to 
weather  information  when  making  aeronautical  deci¬ 
sions  about  flight  over  varying  terrain.  It  replicates 
exploratory  research  (Driskill,  Weissmuller,  Quebe, 
Hand,  Dittmar,  and  Hunter,  1997),  which  was  based 
on  data  from  a  single  geographic  area,  that  provided 
data  tentatively  confirming  three  hypotheses:  First, 
cognitive  processes  pilots  utilize  in  making  aeronauti¬ 
cal  decisions  can  be  modeled  using  regression  meth¬ 
ods;  second,  the  values  pilots  associate  with  varying 
levels  of  ceiling,  visibility,  and  precipitation  are  a 
function  of  the  terrain  over  which  the  flight  is  made; 
and  third,  while  values  differ  among  pilots,  specific 
policies  can  be  found  to  describe  how  they  assign 
weights  in  making  decisions  about  a  flight. 

Reported  are  results  from  data  collected  from  pilots 
from  six  geographical  regions.  The  report  includes  the 
following:  identification  of  worth  functions  pilots 
attribute  to  weather  variables,  description  of  differ¬ 
ences  of  policy  decisions  with  respect  to  the  worth 
functions,  assessment  of  regional  differences  in  use  of 
weather  information,  analysis  of  the  relationship  of 
pilot  demographic  information  and  pilot  decision¬ 
making  with  regard  to  weather  and  terrain  informa¬ 
tion,  comparison  of  results  with  the  exploratory 
research  findings,  assessment  of  pilot  decision  strate¬ 
gies,  and  a  summary  of  findings  and  conclusions. 

II.  BACKGROUND 

The  importance  of  understanding  the  cognitive 
processing  of  flight  information  in  preparing  for  and 
during  flight  is  evident  from  research  reported  by 
Jensen  and  the  National  Transportation  Safety  Board 
(NTSB).  According  to  Jensen  (1982),  80  to  85  per¬ 
cent  of  aircraft  accidents  can  be  assigned  to  pilot  error 
and,  further,  over  50  percent  of  the  fatal  accidents 
result  from  faulty  decision-making.  The  NTSB,  in  a 


study  of  361  small  aircraft  accidents  in  which  583 
fatalities  were  reported  for  276  of  them  (NTSB/SR- 
89/01,  July  14,  1989,  p.l),  reported  that  flight  under 
visual  meteorological  rules  (VFR)  into  instrument 
meteorological  conditions  (IMC),  and  pilot  error  are 
two  of  the  most  important  factors  in  small  aircraft 
fatal  accidents. 

While  considerable  research  is  reported  about  the 
potential  effects  of  Five  Hazardous  Thought  Patterns 
on  decision-making  (see,  for  example,  Jensen,  Adrion, 
6c  Lawton,  1987)  in  aviation  accidents,  very  little  is 
reported  on  the  cognitive  aspects  of  aeronautical  de¬ 
cision-making,  i.  e.,  on  how  pilots  assess  adverse 
conditions  and  assign  and  integrate  appropriate  weights 
to  such  flight  data  elements  as  weather  and  terrain  in 
making  their  judgments.  Two  studies  are  reported  by 
Curry  (1976)  and  Flathers,  Giffin,  and  Rockwell 
(1982),  who  noted  differences  in  worth  functions  or 
values  pilots  assigned  to  various  flight  data  elements 
in  the  landing  and  en  route  phases  of  flight.  In 
addition,  studies  at  the  University  of  Illinois  (Barnett, 
1989;  Stokes,  Kemper,  &  Marsh,  1992;  Wickens, 
Stokes,  Barnett,  &c  Davis,  1987)  have  examined  the 
relationship  of  pilot  biographical  and  demographic 
information,  along  with  knowledge  and  ability  mea¬ 
sures,  to  pilot  performance  and  information  process¬ 
ing  during  simulated  flight  performance. 

Whether  or  not  worth  functions  can  be  modeled 
using  regression  methods  was  the  focus  of  research 
reported  in  Driskill  et  ai,  (1997).  The  approach 
involved  collection  of  pilot  judgments  of  scenarios 
describing  flight  over  non-mountainous,  over  moun¬ 
tainous,  and  over-water  terrain  under  various  visibil¬ 
ity,  ceiling,  and  precipitation  conditions.  Each  scenario 
consisted  of  a  proposed  cross-country  VFR  flight  of 
approximately  one-hour  duration  in  a  single-engine 
aircraft  with  an  inoperative  navigation  radio  and 
without  a  transponder  over  each  of  the  three  kinds  of 
terrain  under  differing  VFR  into  IMC  weather  condi¬ 
tions.  The  procedure  and  findings  from  that  study  are 
summarized  below,  however  readers  are  directed  to 
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the  original  report  (Driskill  et  al.,  1997)  for  a  com¬ 
plete  description. 

A  set  of  27  scenarios  was  developed  for  each  terrain 
type.  Specific  routes  were  drawn  on  route  maps  for 
flight  from  Hereford  airport  to  Dalhart  airport  in  the 
Texas  Panhandle  (non-mountainous),  from  Las  Ve¬ 
gas  airport  to  San  Juan  airport  in  New  Mexico  (moun¬ 
tainous),  and  from  City  County  airport  over  Lake 
Michigan  to  Delta  County  airport  (over  water).  Sce¬ 
narios  were  described  on  5  x  7  cards.  Each  card  in  a  set 
described  different  combinations  of  visibility,  ceiling, 
and  precipitation  conditions,  developed  with  the  as¬ 
sistance  of  National  Oceanographic  and  Atmospheric 
Agency  (NOAA)  and  U.  S.  Air  Force  weather  experts 
to  assure  realistic  weather  combinations,  i.  e.,  that 
combinations  could  be  observed  in  nature.  Maps 
denoting  the  routes  pilots  were  to  follow  and  other 
information  pertinent  to  flight  over  each  terrain  were 
provided  for  reference. 

Pilot  judgments  of  their  level  of  comfort  in  flying 
under  each  of  the  scenario  conditions  were  the  basis 
for  assessing  worth  functions.  The  judgment  task 
required  two  actions  on  the  part  of  the  subject.  First, 
each  pilot  rank  ordered  the  scenarios  in  a  set  in 
descending  order  based  on  how  comfortable  he  or  she 
would  feel  in  flying  over  the  specified  terrain  under 
the  weather  conditions  described.  Second,  using  the 
set  of  rank-ordered  scenarios,  each  pilot  then  assigned 
a  “comfort”  rating  to  each  scenario,  using  the  accom¬ 
panying  comfort  rating  scale.  This  scale  ranged  from 
0,  least  comfortable,  to  100,  most  comfortable.  Pilots 
were  not  constrained  (nor  were  they  expected)  to  rate 
their  most  comfortable  scenario  100.  Instead,  they 
could  assign  any  lesser  level  of  rating  that  described 
their  comfort  level.  Many  of  the  subjects,  in  fact,  used 
a  rating  lower  than  100  to  indicate  their  “most  com¬ 
fortable”  scenario.  Each  pilot  completed  three  decks 
of  scenario  cards,  one  deck  for  each  of  the  terrain 
conditions. 

Most  of  the  data  collected  from  131  male  and  19 
female  pilots  came  from  group  administration  during 
meetings  of  the  FAA  Safety  Seminar  Program  held  in 
the  South  Texas  area  (roughly  from  San  Antonio, 
Austin,  and  Killeen,  south  to  Victoria,  and  west  to 
Rockport).  After  distributing  the  three  decks  of  sce¬ 
nario  cards  (one  deck  for  each  kind  of  terrain),  the 


experimenter  presented  a  motivational  introduction, 
and  then  briefed  and  walked  the  pilots  through  the 
rating  process,  using  sample  scenarios.  The  experi¬ 
menter  remained  through  the  data  collection  process 
to  provide  further  instruction  or  clarification  that 
may  have  been  required. 

Analysis  was  based  on  computation  of  a  multiple 
correlation  for  each  subject  for  each  terrain  condition. 
The  comfort  ratings  a  pilot  assigned  to  the  scenarios 
within  a  terrain  condition  comprised  the  dependent 
variable.  Independent  variables  were  ratings  from  22 
experienced  pilots  on  16  ceiling  levels  ranging  from 
600  to  5,000  feet,  12  visibility  levels  ranging  from  1 
1/2  NM  to  more  than  8  NM,  and  eight  precipitation 
levels  ranging  from  freezing  rain  to  no  precipitation. 
Using  a  0-100  point  scale,  the  experts  rated  how  safe 
it  would  be  for  an  inexperienced  pilot  to  fly  under 
each  of  the  levels  (Driskill  et  ai,  1997).  Multiple 
correlations  for  the  each  of  the  150  pilots  comprising 
the  sample  met  at  least  the  .05  level  of  statistical 
significance. 

Regression  weights  (from  the  individual  multiple 
correlations)  associated  with  the  visibility,  ceiling, 
and  precipitation  variables,  which  represent  the  value 
or  worth  pilots  associated  with  each  of  the  variables, 
were  hierarchically  clustered.  The  analysis  package 
used  was  HIER-GRP  (Appendix  A),  which  is  part  of 
an  Armstrong  Laboratory  mathematical-statistical 
package.  This  analytical  process  formed  clusters  of 
pilots  with  similar  within-cluster  regression  weights; 
between  cluster  weights  (within  terrain  type),  how¬ 
ever,  differed  significantly  with  respect  to  the  value  or 
worth  functions  assigned  to  visibility,  ceiling,  and 
precipitation  information. 

Hierarchical  clustering  produced  four  significantly 
different  non-mountainous  terrain  policy  clusters  or 
groups;  four  groups  for  flight  over  the  mountainous 
route;  and  two  groups  for  the  over-water  route.  Within 
terrain  group  policy  differences  seemed  to  vary  accord¬ 
ing  to  how  conservative  the  subjects  comprising  each  of 
the  groups  were  in  making  their  ratings.  Some  policy 
groups  were  formed  by  pilots  who  provided  relatively 
low  overall  comfort  levels  even  when  flight  was  under  the 
best  weather  conditions  while  some  other  policy  groups 
exhibited  considerably  higher  comfort  levels  for  flight 
under  all  conditions,  including  the  most  adverse. 
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Some  significant  differences  of  demographic  data 
for  clusters  within  a  terrain  condition  were  found, 
mostly  based  on  kinds  of  flying  experience,  age,  and 
aircraft  ownership.  Such  differences,  however,  were 
not  consistent  across  clusters  within  a  terrain  condi¬ 
tion.  Age  of  pilots  comprising  each  of  the  four  clusters 
for  the  non-mountainous  clusters,  for  example,  did 
not  differ,  but  age  in  one  or  more  of  the  clusters  in  the 
other  two  terrain  types  differed  significantly  from 
other  clusters.  Further  analysis  of  the  background 
information  revealed  that  pilots  changed  group  policy 
membership  from  one  terrain  to  another,  i.  e.,  some 
pilots  who  were  members  of  a  policy  group  reporting 
overall  low,  conservative  comfort  levels  changed  group 
membership  to  policy  groups  under  other  terrain 
conditions  having  overall  higher  comfort  levels. 

Analysis  also  addressed  the  decision  strategies  pi¬ 
lots  employed  in  making  their  comfort  level  ratings. 
This  analysis  was  undertaken  because  of  decision 
theory  research  which  suggests  that  judges  may  em¬ 
ploy  different  decision-making  strategies  in  judgment 
tasks  such  as  those  employed  in  the  exploratory  re¬ 
search  (Payne,  Bettman,  &  Johnson,  1992).  Further, 
it  is  important  to  determine  whether  some  pilots  may 
employ  inappropriate  strategies  when  assessing  flight 
information,  so  that  training  or  interventions  could 
be  used  to  redirect  their  decision-making.  In  general, 
pilots  tended  to  use  a  compensatory  strategy  (as  op¬ 
posed  to  multiplicative  or  non-compensatory)  whereby 
good  conditions  in  one  weather  variable  compensate 
for  poorer  conditions  in  other  variables. 

The  fact  that  the  sample  of  pilots  in  the  exploratory 
research  was  drawn  from  pilots  in  the  South  Texas 
area  limits  the  generalization  of  results  to  other  geo¬ 
graphical  regions.  Subsequent  sections  of  this  report 
describe  use  of  the  scenario-based  judgment  task  to 
collect  judgments  from  pilots  from  six  other  geo¬ 
graphical  regions  of  the  United  States. 

III.  METHOD  AND  DATA 
COLLECTION 

The  logistics  of  data  collection  from  a  pilot  sample 
nationwide  required  a  change  from  a  procto red-group 
to  a  mail-out-and-return  protocol.  In  turn,  mail  ad¬ 


ministration  necessitated  modification  of  the  format 
of  the  data  collection  instruments. 

Format  Modifications 

For  efficient  mail  administration,  the  scenario  de¬ 
scriptions  were  transcribed  from  cards  to  answer  sheets. 
Only  the  format  of  the  scenario  information  was 
changed.  The  same  route  maps  were  provided.  Some 
minor  modification  of  the  Pilot  Background  Data 
form  on  which  pilots  provided  demographic  informa¬ 
tion  was  made.  Gender,  for  example,  was  omitted  and 
a  few  additional  informational  items  were  added. 
Instructions,  demographic  information  forms,  and 
answer  sheets  containing  scenario  information  and 
places  for  respondent  rankings  and  ratings  (Appendix 
B)  were  packaged  for  mailing. 

Modifications  of  the  scale  used  and  instructions  for 
making  ratings  were  made  based  upon  comments 
received  from  pilots  during  a  pre-test  of  the  instru¬ 
ment.  In  the  original  study,  the  scale  ranged  from  100 
for  the  most  comfortable  scenario  to  0  for  the  least 
comfortable.  This  scale  was  reversed  for  mail  admin¬ 
istration  with  100  to  be  used  for  the  least  comfortable 
and  0  for  the  most  comfortable.  For  analysis  of  the  data, 
pilot  responses  were  transposed  to  match  the  original 
scale. 

Instructions  for  making  comfort  level  ratings  dif¬ 
fered  also.  In  the  exploratory  study,  respondents  were 
instructed  that  the  maximum  rating,  e.  g.,  100,  did 
not  have  to  he  used  for  the  terrain  scenario  for  which 
they  felt  most  comfortable.  In  this  study,  such  in¬ 
structions  were  not  provided  and,  as  shown  in  Section 
IV,  respondents  appeared  to  anchor  their  ratings  on 
the  extreme  ratings,  e.  g.,  at  0  and  100. 

Pilot  Sample  Selection  and  Description 

The  sample  was  randomly  selected  from  FAA 
records.  Only  pilots  who  were  actively  flying  as  deter¬ 
mined  by  the  presence  of  a  current  flight  physical  record 
in  their  FAA  records  file  were  considered  for  selection. 
Pilots  from  the  South  Texas  area  were  excluded.  Data 
collection  packages  were  mailed  to  1,200  pilots  in  the 
Alaskan,  Western  Pacific,  Southern,  Northwest  Moun¬ 
tain,  Eastern,  and  New  England  regions.  Packages 
contained  a  pre-addressed  and  post-paid  envelope  for 


3 


the  raters  to  return  their  responses  to  the  FAA  in 
Washington,  DC. 

Responses  from  326  respondents  were  received  and 
submitted  for  analysis.  Answer  sheets  were  transcribed 
into  EXCEL  and  a  floppy  disk  containing  the  data  was 
prepared  by  the  FAA  for  analysis. 

Demographic  statistics  describing  the  sample  of 
326  respondents  are  shown  in  Table  1.  Since  not  all 
pilots  completed  all  the  scenarios,  there  were  319 
respondents  for  over- water,  315  for  over  non-moun- 
tainous,  and  303  over  mountainous  terrain.  The  most 
noteworthy  demographic  characteristic  is  the  52-year 
average  age.  This  average  is  almost  10  years  higher 
than  the  average  in  data  compiled  by  the  FAA  (see 
Table  13,  U.S.  Civil  Airmen  Statistics,  1993).  In  this 
report,  the  average  ages  for  private,  commercial,  and 
air  transport  pilots  were  42.7,  41.9,  and  44. 1,  respec¬ 
tively.  Because  of  the  10-year  difference,  demograph¬ 
ics  of  the  present  sample  were  compared  with  a 
mail-administered  nationwide  sample  reported  in 
Hunter  (1995).  As  Table  1  shows,  the  demographics 
of  the  two  mail-administered  samples  are  quite  simi¬ 
lar,  including  the  ages  of  the  respondents. 

The  distribution  of  pilots  among  six  geographical 
regions  is  shown  in  Table  2  while  Table  3  shows  the 
background  variables  for  which  regions  differ  signifi¬ 
cantly  (p  <  .05)  ‘  from  the  Total  Sample.  As  shown,  the 
sample  of  Alaskan  pilots  was  considerably  smaller 
than  were  the  samples  from  other  regions. 

Mean  comfort  level  ratings  were  also  computed, 
using  GRPREL  from  the  Comprehensive  Occupa¬ 
tional  Data  Analysis  Programs  (CODAP)  (see  Appen¬ 
dix  A),  for  the  total  sample  and  for  each  terrain  type. 
The  grand  mean  (total  sample)  was  39.52,  over- water 
39.80,overnon-mountainous,  42.27,  and  over  moun¬ 
tains,  36.38.  The  comfort  rating  for  over-water  terrain 


is  slightly  higher  than  the  grand  mean.  Non-Moun- 
tainous  average  comfort  level  is  about  three  points 
higher  than  the  overall  mean;  and,  the  mountainous 
mean  is  over  three  points  lower. 

IV,  ANALYSIS  OF  REGIONAL  DATA 

Data  analysis  replicated  the  method  used  in  the 
exploratory  research:  interrater  agreement,  policy  sta¬ 
bility,  computation  of  multiple  correlations  for  each 
pilot  respondent,  identification  ofwithin-terrain  policy 
groups  based  on  hierarchical  clustering  of  regression 
weights  associated  with  the  weather  variables  for  each 
terrain  type,  testing  of  significance  of  differences  of 
demographic  variables  among  policy  groups,  and  iden¬ 
tification  of  decision  strategy  pilots  employed  when 
they  made  their  judgments.  Other  analyses  include 
the  effect  terrain  exerts  on  comfort  level,  comparisons 
of  regional  comfort  level,  identification  of  the  sce¬ 
narios  for  each  terrain  for  which  pilots  indicated  the 
most  and  least  comfort,  and  determination  of  relation¬ 
ships  of  demographic  information  to  comfort  level. 

Interratcr  Agreement  and  Stability 

Interrater  agreement  and  stability  of  ratings  for  the 
total  group  and  for  each  regional  group  by  terrain  type 
(Table  4)  was  computed,  using  GRPREL  (See  Appen¬ 
dix  A  for  a  description  of  this  analysis  package). 
Interrater  agreement  is  indicated  by  r^^  and  indi¬ 
cates  the  stability  of  the  policies  represented  by  the 
ratings.  An  r^^  of  .20  is  considered  an  acceptable  level 
of  interrater  agreement  and  any  above  .90  repre¬ 
sents  high  stability.  (The  authors  recognize  that  these 
analysis  procedures  are  little  utilized  outside  military 
occupational  analysis  settings.  For  more  detailed  in¬ 
formation  on  the  procedures  and  their  application. 


’  In  Table  3,  and  Tables  14,  15,  and  17  which  will  follow  later,  caution  is  needed  in  interpreting  the  results.  Because  a  large  number  of 
comparisons  are  being  made  in  each  of  these  tables,  it  is  likely  that  some  of  the  observed  differences  reached  statistical  significance  purely  by 
chance.  We  have  chosen,  in  order  not  to  inflate  the  Type  II  error  rate,  not  to  make  an  adjustment  to  the  individual  comparisons  so  as  to  restrict 
the  experimentwise  error  rate  to  the  nominal  0.05  level.  The  impact  of  this  decision  is  to  place  slightly  more  burden  on  the  reader,  who  must 
keep  in  mind,  for  example,  that  of  the  174  comparisons  made  in  Table  3,  approximately  9  (.05  x  174)  would  reach  statistical  significance 
even  if  there  were  absolutely  no  differences  among  the  regions  on  any  of  the  demographic  items.  Clearly,  since  there  were  37  comparisons 
that  were  found  to  be  statistically  significant  in  that  table,  there  are  some  underlying  differences  among  the  regions.  However,  which  of  the 
37  comparisons  truly  reflect  real  differences  and  which  are  spurious  statistical  artifacts,  we  cannot  say.  Since  our  interest  is  observing  the  general 
pattern  of  differences  (examine,  for  example,  the  flying  experience  of  pilots  in  the  Northwest  Region),  we  do  not  find  this  to  be  a  serious 
restriction.  Rather,  adopting  more  stringent  significance  levels  for  each  individual  comparison,  so  as  to  maintain  the  experimentwise  error 
rate,  would  mask  much  of  the  underlying  richness  of  the  data  by  inflating  the  Type  II  error  rate.  On  balance,  the  current  approach  which 
reports  individual  comparisons  somewhat  liberally  seems  the  most  prudent  course,  keeping  in  mind  that  any  particular  individual  comparison 
should  be  regarded  skeptically. 
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the  reader  may  consult  Albert,  et  al.,  1994;  Christal  & 
Weissmuller,  1976,  1988;  Phalen  &  Mitchell,  1993; 
Staley  &  Weissmuller,  1981;  and  Weissmuller,  Phalen, 
&  Tartell,  1 997.)  Interrater  agreement  across  the  total 
group  of  pilots  and  within  each  geographical  region 
(r^j  )  ranged  from  .31  to  .58.  Stability  was  high, 
ranging  from  .82  to  .99.  In  the  table,  k  shows  the 
average  number  of  raters  per  scenario.  Fractional 
entries  for  the  number  of  raters  are  given  because  the 
software  computes  an  overall  or  grand  mean  number 
of  raters  per  scenario.  For  the  Western  region  overwater 
route,  for  example,  inspection  of  the  number  of  raters 
for  each  scenario  shows  that  46  of  the  pilots  rated  14 
scenarios  and  47  pilots  rated  13  scenarios;  thus  the 
average  number  of  raters  per  scenario  was  46.48, 
rounded  to  46.5.  The  actual  number  of  pilots  rating 
a  scenario  was  used  to  compute  the  scenario  mean 
comfort  level.  The  means  of  the  comfort  level  for  the 
27  scenarios  were  the  basis  for  computing  the  regional 
grand  mean  comfort  rating  over  all  scenarios. 

Policy  Group  Identification  and  Description 

The  high  interrater  agreement  found  for  pilot  com¬ 
fort  level  judgments  for  each  terrain  type  indicates 
that  the  judgments  were  based  on  a  single,  homoge¬ 
neous,  and  common  terrain-specific  policy  with  re¬ 
spect  to  a  terrain  type.  The  pilots,  in  effect,  defined 
three  policies,  one  for  each  terrain  type  by  their 
agreement  about  the  ordering  of  the  comfort  level 
they  would  experience  in  making  flights  under  the 
terrain-weather  variable  scenarios.  Subpolicies  within 
an  overall  terrain  policy,  if  any,  would  be  character¬ 
ized  by  subtle  differences  among  weather-variable 
worth  functions  from  which  comfort  level  ratings 
derived.  Further  analyses  were  required  to  determine 
if  there  were  any  such  differences  of  weights  for  the 
weather  variables  sufficient  to  define  one  or  more 
subpolicies  within  a  terrain  policy.  To  this  objective, 
a  multiple  regression  equation  for  each  pilot  was 
computed  and  the  worth  functions  (regression  weights) 
for  each  variable  were  hierarchically  clustered,  using 
HIER-GRP.  This  clustering  methodology  has  the 
power  to  detect  subtle  differences  of  the  kind  antici¬ 
pated  when  there  is  such  high  interrater  agreement 
and  to  cluster  the  pilots  attributing  similar  weights  to 
the  weather  variables. 


Worth  Function  Computation.  To  identify  the 
worth  functions  (i.  e.,  the  relative  emphasis)  pilots 
attributed  to  the  ceiling,  visibility,  and  precipitation 
variables,  coefficients  for  each  variable  were  com¬ 
puted  for  each  pilot  by  terrain  type  using  the  follow¬ 
ing  regression  equation: 

CF  =  A  +  B  +  C  +  AB  +  AC  +  BC  +  ABC 

where  CF  was  a  vector  of  a  pilot’s  27  weather 
scenario  “comfort”  ratings  for  a  given  type  of  terrain; 
and  A,  B,  and  C  and  their  interactions  were  vectors  of 
benchmark  values  for  ceiling,  visibility,  and  precipita¬ 
tion,  respectively.  The  regression  equations  represent 
a  pilot’s  worth  function  with  respect  to  the  given 
terrain  type.  A  total  of  937  regression  equations  were 
computed  (one  for  each  pilot  for  each  terrain  type),  319 
equations  for  over  water,  3 1 5  for  non-mountainous,  and 
303  for  mountainous  (not  all  326  respondents  com¬ 
pleted  each  terrain  set). 

Individual  R  values  were  high  and  all  statistically 
significant  (p  <  0.05;  null  hypothesis  R  =  zero).  Over¬ 
water  values  were  lowest;  only  47  percent  exceeded 
.70.  Seventy- three  percent,  however,  exceeded  .60. 
For  non-mountainous  terrain,  70  percent  were  .70  or 
higher,  and  for  mountainous  terrain,  84  percent  ex¬ 
ceeded  .70.  Average  R^s  were  .66,  .73,  and  .77,  respec¬ 
tively.  A  large  part  of  the  variance  associated  with  the 
pilots’  ratings  is  explained  by  their  worth  functions 
and  scenario  benchmark  values.  Average  R^s  and  re¬ 
gression  weights  for  the  weather  variables  are  shown 
for  each  terrain  in  Table  5. 

Subpolicy  Group  Specification.  Hierarchical  clus¬ 
tering  revealed  that  each  terrain  policy  was  composed 
of  four  subpolicies  differing  with  respect  to  weather- 
variable  worth  function  emphasis  (i.  e.,  differences  of 
regression  weights) .  Tables  6, 7,  and  8  display  descrip¬ 
tive  data  for  each  of  the  subpolicies,  showing  number 
of  pilots  in  each  group,  average  comfort  level,  average 
R^,  and  highest  regression  weights  (only  weights  > 
+0.20  or  <  -0.20  are  shown  in  the  Tables). 

Comparison  of  the  numbers  of  pilots  in  the  policy 
groups  reveals  that  terrain  type  differentially  affects 
group  membership  size.  For  the  over-water  route,  the 
largest  N  is  139  for  the  policy  group  (W-2)  having  the 
second  lowest  comfort  level  and  a  relatively  small 
number  (24  for  Group  W-1)  constitutes  the  group 
with  the  lowest  mean  comfort  level.  For  the  non- 
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mountainous  route,  the  Ns  decrease  in  order  from  the 
group  {NM-4)  with  the  highest  comfort  level  to  the 
group  (NM-1)  with  the  lowest.  This  pattern  changes 
for  the  over-mountains  route  where  the  largest  num¬ 
ber  is  found  in  the  group  (M-1)  with  the  lowest 
average  comfort  level.  In  terms  of  terrain  effect,  the 
pilots  tended  to  be  more  conservative  in  their  ratings 
of  comfort  level  for  over  mountains  flight.  They 
became  less  conservative  for  over- water  and  over  non- 
mountainous  routes,  respectively. 

Between  subpolicy  group  comparisons  of  mean 
comfort  level  differences  were  all  significant  (p  <  .05 
or  less).  For  the  over-water  route,  mean  ratings  range 
from  13.70  to  55.29,  non-mountainous  from  20.85 
to  54.53,  and  mountainous  from  21 .61  to  53.72.  The 
mean  levels  by  policy  groups  within  each  terrain  type 
are  illustrated  in  Figure  1  in  terms  of  deviation  from 
the  grand  mean  (39.52). 

Average  indicates  the  degree  to  which  a  policy 
group  seems  to  make  use  of  the  experts’  benchmarked 
attribute  values  and  combine  them  in  a  consistent  and 
rational  manner.  Even  though  weights  differ  signifi¬ 
cantly  from  policy  group  to  policy  group  with  respect 
to  ceiling,  visibility,  and  precipitation  variables,  these 
factors  were  very  important  determinants  for  pilots 
when  they  rated  their  comfort  levels,  accounting  for 
most  of  the  variance  (60  to  74  percent)  associated  with 
comfort  level  ratings. 

Regression  weights  in  Tables  5,  6,  7,  and  8  are 
displayed  to  demonstrate  how  groups  differed  with 
respect  to  the  weighting  of  the  weather  variables.  As 
the  tables  show,  some  type  of  interaction  (i.e.,  A*B*C, 
A*B,  A*C,  B*C)  accounts  for  the  larger  regression 
weights.  For  six  of  the  nine  subpolicy  groups  having 
the  lowest  comfort  level  means  (the  more  cautious), 
for  example,  comfort  level  is  primarily  accounted  for 
by  the  three-way  interaction  of  ceiling,  visibility,  and 
precipitation  (A*B*C).  The  three  subpolicy  groups 
having  the  highest  comfort-level  means  (W-4,  NM-4, 
M-4)  have  a  slightly  different  pattern  of  weights.  The 
largest  over-water  subpolicy  weights  are  for  the  two- 
way  interaction  of  ceiling  and  precipitation  (A*C)  and 


visibility  and  precipitation  (B*C);  for  non-mountain¬ 
ous,  two-way  interaction  of  ceiling  and  precipitation 
(A*C)  and  visibility;  and  for  over  mountains,  precipi¬ 
tation  (C)  and  ceiling  and  visibility  (B*C)  interaction. 

The  ceiling- visibility-precipitation  interactions  can 
be  discerned,  though  not  easily,  in  a  careful  inspection 
of  the  scenarios  and  associated  mean  comfort  levels 
displayed  in  Tables  9,10,  and  1 1 .  Some  insight  about 
the  weather-condition  interactions  may  be  gained 
from  Figure  2,  showing  ceiling-precipitation  interac¬ 
tion  for  each  terrain  when  visibility  is  held  constant  at 
8  nautical  miles  (NM).  Two  features  should  be  noted. 
First,  the  dramatic  decrease  in  over-water  comfort 
level  from  5000  feet  ceiling  and  no  rain  to  freezing 
rain  at  the  same  ceiling.  Second,  the  shapes  represent¬ 
ing  the  ceiling-precipitation  interactions  for  over¬ 
water  and  non-mountainous  routes  are  essentially 
alike.  It  also  seems  likely  that  the  shape  for  the 
mountainous  route  would  be  more  similar  if  the 
scenarios  for  that  route  had  contained  precipitation 
conditions  for  ceiling  levels  between  1000-5000  feet. 

Policy  Group  Demographic  Information 

Demographic  information  (from  the  Pilot  Back¬ 
ground  Data  form)  for  policy  groups  is  summarized  in 
Tables  12,  14,  and  16.  Demographic  information 
about  the  total  terrain  group  is  included  with  each  of 
the  three  tables  under  the  headings  TOTALW  (for 
o  ver- water),  TOTALNM  (for  non-mountainous),  and 
TOTALM  (for  mountainous)  for  reference  purposes. 
Means  for  terrain,  precipitation,  visibility,  and  ceiling 
are  based  on  the  5-point  scale  shown  on  the  Pilot 
Background  Data  form  (Appendix  B).  The  scale  was 
1,  “Never;”  2,  “Once  or  Twice;’  3,  “Several  Times;”  4, 
“Many  Times;”  5,  “Numerous  Times.” 

Significant  differences  among  groups,  computed 
from  t-tests  (.05  level)^  of  policy-group  to  policy- 
group  comparisons,  are  detailed  in  Tables  13,15,  and 
17.  Across  the  various  comparisons,  the  most  consis¬ 
tent  differences  seem  to  be  associated  with  experience 
in  flying  over  the  terrain  types  and  personal  ceiling.  In 
general,  as  group  comfort  level  increases,  terrain 


^  Readers  are  reminded  of  the  earlier  caution  regarding  interpretation  of  these  comparisons.  Some  of  the  individual  comparisons  which  attained 
statistical  significance  may  be  spurious  simply  as  a  matter  of  chance.  Patterns  of  relationships,  rather  than  individual  comparisons,  should 
therefore  be  the  principal  focus. 
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experience  tends  to  increase  and  personal  ceiling  tends 
to  decrease. 

Multiple  regression  methodology  was  used  for  two 
purposes.  First,  we  examined  the  degree  to  which  pilot 
comfort  level  was  predicted  from  demographic  vari¬ 
ables.  The  obtained  was  0.156.  Secondly,  we  as¬ 
sessed  the  degree  to  which  pilots  used  weather 
information  consistent  with  the  ratings  by  the  expert 
pilots.  Two  regression  equations  were  computed;  one 
which  used  demographic  variables  and  pilot  comfort 
ratings  to  predict  individual  R^s,  and  one  which  used 
only  demographic  variables  as  predictors.  The  R^s 
were  0.176  and  0.149,  respectively.  Results  of  the 
regression  analyses  are  displayed  in  Appendix  C,  which 
shows  that  zero-order  correlations  (ZOC)  and  unique 
contribution  of  each  of  the  variables  to  the  separate 
criteria  are  small.  These  results  indicate  that  age, 
certification  and  flying  experience  have  little  influ¬ 
ence  on  pilots’  ratings  of  comfort  level.  Further,  in 
view  of  the  high  individual  multiple  Rs  reported  in  an 
earlier  section,  the  worth  functions  pilots  attributed 
to  the  weather  variables  (1)  accounted  for  most  of  the 
variance  in  the  pilots’  comfort  ratings  and  (2)  were 
consistent  with  expert  pilot  ordering  of  the  impor¬ 
tance  of  the  weather  variables. 

Regional  Comparisons 

Analyses  addressed  differences  of  mean  comfort 
level  among  regions  by  terrain  type,  regional  worth 
function  comparisons,  demographic  characteristics, 
and  the  proportionality  of  regional  membership  among 
subpolicy  groups.  In  reviewing  these  analyses,  readers 
should  bear  in  mind  the  limited  sample  of  Alaskan 
pilots  (N  =  13),  which  makes  interpretation  of  results 
for  that  region  problematic. 

Mean  Comfort  Levels.  The  mean  comfort  levels 
for  each  region  by  terrain  type  are  given  in  Table  18. 
Significant  (p  <  .05)  differences  in  the  mean  comfort 
levels  among  the  regions  for  each  terrain  type  are 
summarized  below: 

Over-Water  Terrain 

•  Northwest  Mountain  region  is  significantly  lower 
than  Total  Group,  Southern  region.  Western  Pa¬ 
cific  region  means 


•  Southern  region  is  significantly  higher  than  Total 
Group  mean 

Non-Mountainous  Terrain 

•  Alaska  region  mean  is  significantly  higher  than  the 
Total  Group  and  all  other  regional  means 

•  Northwest  Mountain  region  is  significantly  lower 
than  Total  Group  mean 

Mountainous  Terrain 

•Alaska  region  mean  is  significantly  higher  than 
Total  Group  and  all  other  regional  means 

Worth  Functions.  Average  and  regression 
weights  for  the  geographical  regions  by  terrain  type 
have  been  previously  shown  in  Table  5.  Overall,  the 
amount  of  comfort-level  variance  accounted  for  by 
the  expert  ratings  of  the  weather  variables  was  not  only 
higher  for  the  Alaskan  pilots  than  the  total  sample  (all 
terrains)  but  was  higher  than  the  other  regions  for  both 
the  non-mountainous  and  mountainous  terrains. 

For  over-water  terrain,  the  R^s  were  similar  among 
regions,  accounting  for  62  to  70  percent  of  the  variance 
in  comfort  levels.  Except  for  Alaska,  worth  functions 
are  similar  with  the  highest  weight  attributed  to  the 
interaction  of  ceiling,  visibility,  and  precipitation 
(A*B*C).  The  Alaskan  weights  are  somewhat  differ¬ 
ent;  the  largest  weight  is  attributed  to  an  interaction 
of  ceiling  and  precipitation  (A*C)  followed  by  the 
A*B*C  interaction. 

Non-mountainous  terrain  R^s  are  larger  than  those 
for  over  water,  ranging  from  .71  to  .82.  R^  are  the 
product  of  ceiling-visibility  (A*B)  and  ceiling-pre¬ 
cipitation  (A*C)  interactions  for  all  regional  groups. 

Average  R^s  for  the  mountainous  terrain  are  the 
highest  of  all.  Expert  ratings  contribute  from  74  to  79 
percent  of  the  variance  in  comfort  levels.  Generally, 
pilots  attribute  most  weight  to  the  ceiling-visibility- 
precipitation  interaction. 

Regional  Group  Membership  Proportionality.  The 
issue  of  proportionality  of  policy  group  membership 
was  addressed  by  determining  if  membership  in  any 
given  subpolicy  group  was  comprised  of  a  larger  or 
smaller  number  of  pilots  than  should  be  expected 
from  the  proportion  of  pilots  for  the  region  compris¬ 
ing  the  total  sample.  As  shown  in  Table  20, 6  of  the  72 
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comparisons  were  found  to  be  statistically  different 
from  the  mean  proportions  (p  <  .05).  Since  we  would 
have  expected  4  comparisons  to  attain  statistical  sig¬ 
nificance  simply  because  of  chance,  the  results  do  not 
present  strong  evidence  of  differences  in  the  propor¬ 
tionality  of  policy  group  membership.  However,  since 
4  of  the  6  significant  differences  were  found  for 
Alaskan  pilots,  the  data  are  suggestive  (though  cer¬ 
tainly  not  definitive)  of  a  real  difference  in  propor¬ 
tionality  for  that  group.  More  Alaskan  pilots  than 
should  be  expected  were  members  of  the  terrain  groups 
(W-4,  NM-4,  and  M-4)  having  the  highest  comfort 
levels  (55.29  for  over  water,  54.53  for  non-mountain- 
ous,  and  53.72  for  mountainous)  for  each  terrain  type. 

Given  the  results  of  the  analysis  of  regional  mem¬ 
bership  in  subpolicy  groups,  comparisons  of  the  re¬ 
gional  groups  against  the  Total  Group  were  made  (see 
Table  3).  Differences  were  noted  for  19  of  the  30 
experience  items  in  the  Pilot  Background  Data  form. 
Many  of  the  differences  are  in  the  direction  to  be 
anticipated.  That  is,  Alaskan  pilots  reported  more 
experience  with  heavy  snow;  the  Western  Pacific  and 
Northwest  Mountains  pilots  have  more  experience  in 
flight  over  mountains.  While  Table  3  provides  some 
insight  for  the  differences  of  Northwest  Mountains 
membership,  there  is  nothing  to  support  the  Alaskan 
proportionality  findings.  Northwest  Mountain  pi¬ 
lots,  although  reporting  more  experience  over  water 
and  in  the  mountains,  have  significantly  less  experi¬ 
ence  in  all  the  ceiling  and  visibility  conditions.  With 
respect  to  the  Alaskan  region,  it  may  very  well  be  that 
pilots  there  represent  a  different  sample  with  respect 
to  comfort  level  than  found  elsewhere  in  the  United 
States.  Such  an  assumption,  however,  can  only  be 
tentative,  because  of  the  small  Alaskan  sample  size. 

Worst  Flight  Scenario 

Pilots  were  asked  to  choose  from  among  the  sce¬ 
narios  (i.  e.,  the  27  combinations  of  weather  variables) 
the  one  scenario  that  represented  the  worst  flight 
scenario  under  which  they  would  consider  continuing 
the  flight  described  for  each  terrain  type.  Data  for  this 
item,  however,  were  not  usable,  since  some  pilots 
followed  the  instructions,  while  others  simply  listed 
the  scenario  they  had  ranked  worst  for  a  particular 
terrain  type. 


V.  COMPARISON  WITH  SOUTH 
TEXAS  SAMPLE  RESULTS 

There  is  a  striking  similarity  between  the  hierarchi¬ 
cal  clustering  results  from  the  regional  and  South 
Texas  samples  for  flight  over  non-mountainous  and 
mountainous  routes.  Subpolicies  for  each  terrain  con¬ 
dition  w'ere  alike  with  respect  to  how  pilots  viewed 
flight  under  the  ceiling,  visibility,  and  precipitation 
conditions.  By  route  type,  both  samples  were  com¬ 
prised  of  groups  of  pilots  who  made  uniquely  conser¬ 
vative  ratings  of  comfort  level,  even  for  the  best 
combinations  of  conditions,  as  well  as  other  groups 
making  progressively  less  conservative  ratings.  Over¬ 
water  route  results  differed  with  respect  to  number  of 
subpolicy  groups  identified.  Where  there  were  four 
subpolicies  for  the  regional  sample,  there  were  only 
two  for  the  South  Texas  sample.  The  policies  from 
both  samples,  however,  are  characterized  by  progres¬ 
sively  less  conservative  comfort  ratings. 

Less  similarity  is  noted  for  other  factors.  Compari¬ 
sons  of  the  South  Texas  and  regional  samples  for 
demographic  information,  average  comfort  level,  and 
for  average  and  weather  variable  average  regression 
weights  are  described  in  the  following  paragraphs. 

Demographic  Information 

Comparison  of  the  data  from  the  326  pilots  in  six 
geographical  regions  with  the  150  pilots  from  South 
Texas  revealed,  first,  the  South  Texas  sample  was  10 
years  younger  and  more  nearly  matches  the  1993 
national  average  age  of  42.7  years  (Table  21).  Second, 
the  South  Texas  pilots  had  fewer  total  flying  hours 
(1694  to  3845  hours  logged)  and  were  less  experi¬ 
enced  in  flight  over  the  three  terrains.  In  addition,  for 
a  large  number  of  the  remaining  variables,  the  South 
Texas  pilots  had  less  experience  in  flying  under  the 
different  ceiling,  visibility,  and  precipitation  levels. 
The  variables  for  which  South  Texas  pilots  reported 
higher  means  are  for  flying  hours  logged  in  the  past  90 
days,  use  of  the  FAA/FSS  weather  source,  and  for 
larger  number  holding  private  pilot  licenses.  A  vari¬ 
able-by-variable  comparison  of  the  two  samples  is 
shown  in  Tables  22,  23,  and  24.  Subpolicy  alignment 
follows  for  the  groups  having  the  lowest  comfort  level 
means  to  a  comparison  of  groups  with  the  highest 
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means.  The  South  Texas  sample  data  are  shown  under 
the  columns  labeled  GP. 

Comfort  Level 

For  all  subpolicy  groups,  the  South  Texas  pilots 
reported  much  lower  mean  comfort  level  ratings.  The 
lower  ratings,  however,  may,  in  part,  be  attributable  to 
the  differences  in  instructions  for  providing  the  ratings 
(see  Methodology  and  Data  Collection,  Section  III). 

Average  and  Average  Regression  Weights 

Regional  average  and  regression  weights  for 
subpolicy  groups  are  displayed  in  Tables  6,  7,  and  8. 
Comparable  information  for  the  South  Texas  subpolicy 
groups  are  shown  in  Driskill  et  al.y  1997.  While 
average  R^s  are  fairly  similar  (that  is,  the  amount  of 
total  variance  in  comfort  ratings  ranges  from  approxi¬ 
mately  60  to  80  percent),  inspection  of  the  two  sets  of 
tables  reveals  different  patterns  in  the  average  weights 
the  pilots  assign  to  ceiling,  visibility,  and  precipita¬ 
tion  and  their  interactions. 

VI,  DECISION  STRATEGY 

To  determine  the  extent  to  which  pilots  used  com¬ 
pensatory  or  noncompensatory  strategies  when  as¬ 
signing  weather  scenario  comfort  ratings,  the 
correlations  between  comfort  ratings  and  sets  of  com¬ 
pensatory  and  noncompensatory  models  were  exam¬ 
ined  and  compared.  Specifically,  the  standardized 
scenario  benchmark  ratings  (which  were  standardized 
to  a  mean  of  5.0  and  a  standard  deviation  of  1 .0  for  all 
scenarios)  were  transformed  (using  sets  of  transforma¬ 
tions  designed  to  simulate  compensatory  and  non¬ 
compensatory  policies)  and  then  correlated  with  the 
pilots’  scenario  mean  comfort  ratings  for  each  terrain 
type.  The  models  consisted  of  additive,  multiplica¬ 
tive,  worst-factor  cut-off,  single-factor,  continuous, 
and  cut-off  models,  one  each  for  ceiling,  visibility, 
and  precipitation. 

Within  the  compensatory  model  set,  the  additive 
model  summed  the  three  benchmark  factors  and  im¬ 
plied  equal  weights  for  the  three  factors.  The  multipli¬ 
cative  model,  both  two  and  three  factor  models, 
cross-multiplied  benchmark  values.  This  model  im¬ 
plies  a  policy  in  which  one  or  more  factors  may 


introduce  a  dampening  effect  which  can  overpower 
the  level  of  the  other  factors. 

The  worst-factor  cutoff  models  used  the  lowest  of 
the  three  benchmark  values.  This  model  implies  a 
policy  in  which  the  poorest  of  the  three  factors  became 
the  focus. 

Noncompensatory  models  were  represented  by 
continuous  single  factors  (A,  B,  or  C)  as  the  decision 
variable  and  single-factor  cut-offs.  A  pilot  using  a 
single  continuous  factor  to  establish  a  cut-off  value 
would  represent  a  policy  in  which  the  focus  was  on 
either  ceiling,  visibility,  or  precipitation  (A,  B,  or  C) 
or  some  established  cut-off  value  (e.  g.,  X  >  4.5)  as  a 
single  judgment  factor. 

Table  25  lists  the  compensatory,  worst-factor,  and 
noncompensatory  models  and  their  respective  corre¬ 
lations  with  the  pilot  comfort  ratings.  This  Table 
shows,  just  as  in  the  case  of  the  South  Texas  sample, 
the  multiplicative  and  additive  models  consistently 
showed  the  strongest  relationships  with  assigned  com¬ 
fort  ratings. 

It  should  be  noted,  however,  that  the  worst-factor 
model  where  X  >  4.5  also  had  relatively  strong  rela¬ 
tionships  with  comfort  ratings.  These  data  suggest 
that  values  less  than  one-half  (.5)  standard  deviation 
below  the  mean  of  the  expert  (benchmark)  rating  for 
any  one  of  the  three  variables  were  used  as  the  deter¬ 
mining  judgment  factor  when  the  pilots  made  their 
comfort  ratings.  That  is,  individual  pilots  tend  to  have 
a  self-defined  level,  a  personal  standard,  for  ceiling, 
visibility,  or  precipitation  that  establishes  the  point  at 
which  they  become  very  uncomfortable  and,  most 
probably,  would  be  reluctant  to  make  a  flight  under 
that  condition. 

VII.  FINDINGS  AND  CONCLUSIONS 

Although  the  six  geographical  region  and  South 
Texas  samples  are  markedly  different  samples  of  the 
continental  United  States  pilot  population,  there  is 
high  substantive  agreement  of  results.  To  the  degree 
that  these  samples  are  representative  of  the  general 
pilot  population,  therefore,  this  agreement  strongly 
suggests  that  the  findings  and  conclusions  can  be 
generalized  to  the  national  pilot  population. 
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In  summary,  this  study  found: 

1 .  Pilot-unique  worth  functions  or  regression  weights 
attributed  to  ceiling,  visibility,  and  precipitation 
conditions  for  differing  terrain  in  aeronautical 
decision-making  can  be  modeled  using  regres¬ 
sion  methods. 

2.  In  the  context  of  the  scenarios  used  in  this  study, 
weather  variable  regression  weights  accounted  for 
a  large  proportion  (66  to  77  percent)  of  the 
variance  associated  with  pilot  decisions  to  make  a 
flight,  as  reflected  in  their  comfort  levels. 

3.  These  worth  functions  consistently  parallel  the 
emphasis  experts  assign  to  flight  under  the  vari¬ 
ous  levels  of  ceiling,  visibility,  and  precipitation 
over  the  three  terrain  types. 

4.  Although  there  appears  to  be  a  common  ordering 
of  the  weather  variables  in  terrain  policy,  each 
pilot  attributes  uniquely  different  weights  to  weather 
conditions  for  each  terrain  type.  Hierarchical 
clustering  of  these  pilot-unique  regression  weights, 
with  respect  to  the  worth  functions  attributed  to  the 
weather  variables,  produced  four  statistically  differ¬ 
ent  policy  groups  of  pilots  for  each  terrain  route. 

5.  Mean  policy  group  comfort  levels,  the  product  of 
the  regression  weights  for  the  weather  variables, 
differ  markedly  among  terrain  subpolicy  groups 
and  range  from  conservative  to  progressively 
higher  levels. 

6.  Demographic  characteristics  of  pilots  forming 
the  subpolicy  groups  account  for  15' 17  percent 
of  policy  variance  at  most.  From  a  training  per¬ 
spective,  this  result  is  disappointing,  because  it  is 
not  possible  to  identify  pilots  for  whom  targeting 
training  interventions  might  be  most  useful. 

7.  The  interactive  effect  of  variation  in  ceiling,  vis¬ 
ibility,  and  precipitation  in  decision-making  is 
complex  and,  except  for  freezing  precipitation 
where  pilots  are  reluctant  to  make  a  flight,  gener¬ 
alization  about  any  one  of  the  variables  is  inap¬ 
propriate.  In  combining  the  weather  variables, 
pilots  tend  to  trade-off  a  poor  value  for  one 
variable  for  good  values  in  others. 

8.  While  pilots  tend  to  use  compensatory  (multipli¬ 
cative  and  additive)  decision  strategies  in  consid¬ 
ering  the  weather  variables,  there  is  evidence  that 
a  worst- factor  strategy  operates  for  some  weather 


scenarios.  That  is,  pilots  appear  to  have  personal 
standards  for  either  ceiling,  visibility,  or  precipi¬ 
tation  below  which  they  become  reluctant  to 
make  a  flight. 

9.  Since  neither  weather  condition  nor  demographic 
variable  weights  individually  nor  cumulatively 
account  for  the  total  variance  associated  with 
pilot  comfort  levels,  other  factors  that  can  affect 
pilot  decision-making  warrant  investigation. 
Other  personal  characteristics  and  the  willing¬ 
ness  of  pilots  to  take  risks  may  be  potential  sources 
of  some  of  the  variance.  In  addition,  the  current 
study  attempted  to  hold  constant  motivational 
factors  —  the  urgency  of  making  a  given  flight  — 
through  the  specification  of  the  flight  scenarios. 
Had  motivational  factors  been  allowed  to  vary, 
different  weighting  and  integration  of  the  weather 
information  may  have  occurred. 

Given  these  findings,  how  then  does  one  make  use 
of  the  results.  First,  let  us  note  that,  on  average,  the 
pilots  agree  with  the  risk  assessments  given  by  experts 
of  the  various  weather  combinations.  This  replicates 
the  previous  finding  (Driskill,  et  al.,  1997)  and  the 
findings  of  a  study  using  a  considerably  different 
assessment  instrument  (Driskill,  et  al.,  in  press).  Given 
that  accidents  are  relatively  rare  events,  it  should  not 
be  too  surprising  that  experts  and  the  general  popula¬ 
tion  of  pilots  share  a  common  view  of  what  constitute 
safe  and  hazardous  weather  conditions.  Still,  there  is 
some  variability  in  risk  assessments,  as  indicated  by 
differences  in  comfort  levels  among  the  hierarchical 
groups  for  each  terrain.  If,  as  this  and  other  studies 
have  found,  demographic  variables  do  not  predict 
differential  comfort  level  (which  may  equate  to  risk 
tolerance),  then  the  present  instrument  may  be  one 
way  to  assess  those  differences. 

The  next  question,  of  course,  is  whether  or  not 
those  differences  in  comfort  level  or  group  member¬ 
ship  have  any  real  utility.  For  example,  are  they 
associated  with  differential  involvement  in  accidents, 
incidents,  or  critical  flying  events?  If  these  measures 
are  found  to  be  valid  predictors  of  those  criteria,  then 
using  them  to  make  pilots  aware  of  their  risk  tolerance 
might  be  worthwhile.  Similarly,  pilot  awareness  of 
their  preferred  decision  model  (i.e.,  compensatory, 
cut-off  levels)  might  also  prove  worthwhile  if  it  led  to 
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a  change  of  behavior  toward  safer  operations.  One 
might  also  speculate  that  low-time,  VFR  pilots  should 
use  a  non-compensatory  model  to  reduce  the  risk  of 
weather  accidents.  All  of  these  issues  are  open  to 
study,  and  may  be  addressed  in  future  studies. 

Additionally,  a  reviewer  (K.  Joseph,  personal  com¬ 
munication,  July  2,  1997)  suggested  that  the  findings 
relating  to  decision  strategy  types  may  be  useful  to 
designers  of  flight  planning  software.  For  example, 
knowledge  about  pilot  decision  strategies  when  con¬ 
sidering  weather  information  could  be  used  to  en¬ 
hance  weather-related  decisions  made  by  pilots  during 
flight  planning.  Flight  planning  software  could  be 
designed  to  include  weather-related  decision  aids  that 
reduce  and  possibly  eliminate  poor  decision  strategies. 

While  the  present  study  has  shown  that  weather- 
related  decision  making  may  be  modeled  successfully 
using  linear  modeling  techniques,  much  remains  to  be 
done  to  determine  whether  this  approach  can  make  a 
significant  contribution  to  improving  aviation  safety. 
Future  studies  will  address  the  relationships  of  mea¬ 
sures  derived  from  this  modeling  technique  with  crite¬ 
ria  of  interest  and  how  best  to  apply  the  results  to  achieve 
a  safety  benefit. 
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Pilot  Comfort  Levels  at  8NM  Visibility 
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Table  1.  1997  Sample  Demographics 
Compared  with  Demographics  in  1995  Study 


Item 

1997 

1995’ 

Mean  Age 

52  yrs. 

50  yrs. 

CFI  or  CFn  certificate 

34% 

34% 

Private  pilot  license 

33% 

38% 

Commercial  license 

44% 

43% 

Airline  transport  license 

20% 

18% 

Multi-engine  rated 

49% 

49% 

Instmment  rated 

77% 

71% 

Employed  as  pilot 

19% 

34% 

Flying  hours  90  days 

36 

29 

Total  flying  hours  logged 

3845 

3465 

Mean  comfort  level  -  total  group 

39.52 

Mean  over-water  comfort  level 

39.80 

Mean  non-mountain  comfort  level 

42.27 

Mean  mountain  comfort  level 

36.38 

’From  Hunter  (1995) 


Table  2.  Sample  Distribution  by  Terrain  Type  and  Geographic  Region 


Terrain 

N 

Alaska 

Northwest 

Water 

319 

12 

56 

Non- 

315 

13 

57 

Mountain 

Mountain 

303 

13 

54 

Western 

Southern 

Eastern 

New 

England 

49 

50 

57 

82 

48 

50 

55 

81 

47 

48 

52 

79 

15 


Table  3.  Mean  Differences  Comparisons  for  Total  and  Geographic  Region  Samples* 


Demograohic  Item 

Total 

AK 

NW 

WP 

SO 

EA 

Age 

52yrs. 

_ 

_ 

CFI  or  CFn  certificate 

34% 

- 

_ 

_ 

_ 

Private  pilot  license 

33% 

- 

_ 

_ 

_ 

Commercial  license 

44% 

_ 

_ 

_ 

Airline  Transport  license 

20% 

- 

- 

_ 

_ 

_ 

Multi-engine  rated 

49% 

- 

- 

_ 

_ 

Instmment  rated 

77% 

- 

- 

_ 

_ 

_ 

Employed  as  pilot 

19% 

- 

H 

- 

H 

L 

Own  aircraft 

52% 

Hours  logged  last  90  days 

36 

- 

- 

H 

L 

Total  hours  logged 

3845 

- 

- 

_ 

_ 

L 

Number  FAA  Seminars 

1.69 

- 

L 

attended 

%  Time  weather  from 

58 

- 

_ 

_ 

FAA/FSS 

%  Time  weather  from 

41 

_ 

_ 

L 

H 

radio/TV 

%  Time  weather  from  DUAT 

26 

_ 

_ 

_ 

_ 

%  Tune  weather  other  sources 

16 

- 

_ 

- 

_ 

_ 

L 

Lowest  ceiling  day  VFR 

2184  ft. 

L 

H 

- 

L 

_ 

Flying  experience^ 

Terrain 

over  water 

2.84 

H 

H 

L 

H 

H 

L 

over  non-mountain 

4.42 

- 

- 

- 

_ 

over  mountains 

3.48 

- 

_ 

_ 

_ 

Precipitation  conditions 

light  rain 

3.94 

- 

- 

- 

H 

H 

L 

moderate  rain 

3.36 

- 

- 

- 

H 

L 

_ 

heavy  snow 

2.14 

H 

- 

- 

_ 

_ 

Visibility  conditions 

1  mile 

3.15 

- 

L 

H 

L 

4  miles 

4.18 

- 

L 

_ 

_ 

8  miles 

4.72 

L 

H 

- 

Ceiling  conditions 

800  feet 

3.14 

- 

L 

_ 

H 

- 

1800  feet 

4.05 

- 

L 

_ 

- 

L 

4000  feet 

4.78 

- 

L 

■  _ 

'Significance  of  differences  computed  by  t-tests  at  0.05  level  for  individual  comparisons 

indicates  significantly  higher  mean  than  Total  Group,  L  significantly  lower  than  Total  Group,  -  no  difference 
^Rating  scale  used:  1,  Never:  2,  Once  or  twice;  3,  Several  times;  4,  Many  times;  5,  Numerous  times 
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Table  4.  Interrater  Agreement  and  Stability  for  Regional 
and  Total  Samples  by  Terrain  Type 


Over  Water 


Region 

Eli 

Rkk 

k 

Alaska 

0.3450 

0.8528 

11.0 

Western 

0.4772 

0.9770 

46.5 

Southern 

0.5177 

0.9810 

48.2 

Northwest 

0.4944 

0.9818 

55.3 

Eastern 

0.5778 

0.9873 

56.8 

New  England 

0.5219 

0.9888 

80.9 

Total 

0.5130 

0.9968 

300.0 

Non-Mountainous 

Regigo 

Rll 

Rkk 

k 

Alaska 

0.4948 

0.9112 

10.5 

Western 

0.5264 

0.9811 

46.9 

Southern 

0.5314 

0.9815 

46.7 

Northwest 

0.5265 

0.9840 

55.3 

Eastern 

0.5506 

0.9851 

53.9 

New  England 

0.4938 

0.9870 

77.8 

Total 

0.5214 

0.9968 

286.9 

Mountainous 

Region 

Rll 

Rkk 

k 

Alaska 

0.3082 

0.8200 

10.2 

Western 

0.5028 

0.9794 

46.9 

Southern 

0.5369 

0.9811 

44.7 

Northwest 

0.5866 

0.9867 

52.3 

Eastern 

0.5521 

0.9837 

49.0 

New  England 

0.5529 

0.9895 

76.5 

Total 

0.5430 

0.9970 

275.9 

17 


Table  5.  Average  Regression  Weights  for  Terrain  Policies 


Over  Water  Non-Mountain  Mountains 


Number  of  Members 

319 

315 

303 

Average  Comfort  Level^ 

39.80 

42.27 

36.38 

Average 

0.66 

0.73 

0.77 

Regression  Weights 

Ceiling  (A) 

-0.41 

-0.38 

Visibility  (B) 

Precipitation  (C ) 

-0.31 

(MULTA^B*C) 

1.48 

0.99 

(MULTA^B) 

-0.54 

0.75 

0.23 

(MULTA^C) 

0.40 

(MULTB*C) 

-0.33 

♦Comfort  level  means  are  significantly  different  (p<.05) 


Table  6.  Over-Water  Subpolicy  Group  Mean  Regression  Weights 


Total 

Group  W-1 

Group  W-2 

Group  W-3 

GrouD  W-4 

TOTALW 

Number  of  Members 

319 

24 

139 

72 

84 

Mean  Comfort  LeveP 

39.80 

13.79 

31.76 

46.73 

55.29 

MeanR^ 

0.66 

0.67 

0.70 

0.61 

0.65 

Regression  Weights 

Ceiling  (A) 

-0.24 

Visibility  (B) 

0.20 

Precipitation  (C  ) 

-0.31 

-0.38 

-0.33 

-0.38 

(MULT  A^B^C) 

1.48 

1.41 

2.05 

1.84 

0.26 

(MULTA^B) 

-0.54 

-0.82 

-0.58 

-0.76 

-0.21 

(MULTA^C) 

0.60 

(MULTB^C) 

-0.38 

-0.41 

0.47 

♦Comfort  level  means  are  significantly  different  {t-test,  p<0.05) 
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Table  7.  Non-Mountainous  Subpolicy  Group  Mean  Regression  Weights 


Total 

Group  NM-1 

Group  NM-2 

Group  NM-3 

TOTALNM 

Number  of  Members 

315 

49 

72 

93 

Average  Comfort 

42.27 

20.85 

38.45 

43.82 

Level* 

Average 

0.73 

0.71 

0.73 

0.74 

Regression  Weights 

Ceiling  (A) 

-0.41 

-0.63 

-0.78 

Visibility  (B) 

-0.20 

-0.28 

Precipitation  (C ) 

0.20 

(MULT  A*B*C) 

-0.29 

1.62 

-0.84 

(MULTA*B) 

0.75 

1.18 

-0.41 

1.93 

(MULTA*C) 

0.40 

0.56 

0.57 

(MULT  B*C) 

-0.23 

*Comfort  level  means 

are  significantly  different  (t-test,  p<0.05) 

Table  8.  Mountainous  Subpolicy  Group  Mean  Regression  Weights 


Total 

TOTALM 

Group  M-1 

Group  M-2 

Grouo  M-3 

Number  of  Members 

303 

94 

78 

63 

Average  Comfort 
Level* 

36.38 

21.61 

35.57 

40.71 

Average  R^ 

0.77 

0.69 

0.62 

0.67 

Regression  Weights 

Ceiling  (A) 

Visibility  (B) 
Precipitation  (C ) 

-0.38 

-0.48 

-0.86 

(MULTA*B*C) 

0.99 

1.53 

1.96 

(MULTA*B) 

(MULTA*C) 

0.23 

0.20 

0.34 

-0.35 

0.75 

1.38 

(MULT  B*C) 

-0.33 

-0.47 

-0.44 

-0.30 

*Comfort  level  means  are  significantly  different  (t-test,  p<.05) 


Group  NM- 
4 

101 

54.53 

0.74 

0.39 

0.27 

0.47 


Group  M-4 

68 

53.72 

0.63 

0.48 

0.46 
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Table  9.  Over-Water  Scenario  Mean  Comfort  Ratings 


No. 

Ceiling  fin  feetl 

Visibility  fin  NMl 

Precipitation 

N 

Mean 

104 

5000 

>8 

none 

318 

90.94 

103 

4000 

4 

none 

318 

83.51 

101 

4000 

7 

moderate  rain 

318 

72.93 

112 

1600 

8 

none 

317 

72.22 

105 

2000 

3 

none 

316 

70.28 

114 

4000 

5 

light  snow 

316 

63.90 

110 

1800 

7 

moderate  rain 

317 

57.02 

107 

1800 

7 

light  snow 

317 

54.80 

125 

1800 

5 

moderate  rain 

314 

51.34 

102 

5000 

IVi 

light  rain 

315 

43.45 

122 

900 

5 

none 

314 

40.06 

106 

1800 

3 

heavy  rain 

313 

37.80 

124 

1600 

4 

moderate  snow 

313 

37.41 

119 

600 

7 

none 

313 

32.99 

123 

800 

4 

moderate  rain 

312 

27.66 

118 

600 

8 

moderate  rain 

313 

25.98 

111 

800 

l*/2 

none 

313 

25.80 

108 

3500 

1 

heavy  rain 

314 

25.63 

113 

1600 

3 

heavy  snow 

315 

25.02 

127 

900 

4 

heavy  rain 

309 

21.08 

120 

1600 

l»/2 

moderate  snow 

313 

20.83 

116 

1600 

l>/2 

heavy  rain 

312 

20.29 

109 

900 

l»/2 

moderate  rain 

314 

20.09 

126 

1800 

l«/2 

none 

314 

18.45 

117 

5000 

11/2 

moderate  rain 

312 

17.50 

115 

700 

IVl 

heavy  snow 

312 

5.69 

121 

5000 

8 

freezing  rain 

313 

4.95 
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Table  10.  Non-Mountainous  Scenario  Mean  Comfort  Ratings 


Na 

Ceiling  fin  feet) 

Visibility  fin  NM) 

Precipitation 

N 

Mean 

206 

4000 

8 

light  rain 

313 

85.18 

201 

5000 

8 

light  snow 

312 

80.27 

226 

2000 

>8 

light  rain 

310 

74.83 

227 

4000 

5 

light  snow 

308 

70.89 

210 

1800 

4 

none 

311 

70.27 

204 

4000 

3 

light  rain 

311 

66.76 

205 

2000 

7 

light  snow 

310 

65.92 

217 

1800 

7 

moderate  rain 

309 

59.86 

213 

1800 

5 

moderate  rain 

311 

57.68 

212 

800 

8 

none 

308 

46.31 

223 

1600 

4 

moderate  snow 

309 

41.21 

220 

2000 

IV2 

none 

305 

40.34 

211 

8000 

>8 

light  snow 

308 

38.19 

202 

4000 

1 

light  rain 

306 

37.5 

224 

800 

3 

none 

308 

36.61 

221 

1800 

3 

heavy  rain 

308 

35.141 

216 

5000 

IV2 

moderate  rain 

304 

34.84 

218 

900 

5 

light  snow 

306 

32 

203 

1600 

3 

heavy  snow 

309 

29.29 

214 

600 

8 

moderate  rain 

303 

28.17 

215 

900 

4 

heavy  rain 

302 

23.16 

225 

900 

P/2 

light  rain 

305 

20.87 

207 

3500 

V2 

heavy  rain 

306 

14.75 

208 

1800 

V2 

moderate  snow 

306 

11.83 

209 

600 

V2 

light  snow 

307 

9.38 

219 

900 

V2 

light  rain 

306 

8.08 

222 

2000 

V2 

freezing  rain 

304 

3.92 
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Table  11.  Mountainous  Scenario  Mean  Comfort  Ratings 


No. 

Ceilins  fin  feet) 

Visibility  lin  NM) 

Precipitation 

N 

Mean 

307 

4000 

8 

none 

298 

86.31 

301 

5000 

5 

none 

298 

84.83 

316 

2000 

>8 

Ught  rain 

295 

67.19 

303 

3500 

>8 

moderate  rain 

295 

66.00 

325 

2000 

4 

none 

295 

64.55 

313 

4000 

4 

Ught  snow 

293 

60.63 

321 

2000 

5 

light  snow 

293 

52.39 

309 

1800 

7 

moderate  rain 

294 

49.95 

327 

1800 

5 

moderate  rain 

292 

46.54 

326 

5000 

l>/2 

light  rain 

292 

39.99 

317 

2000 

11/2 

none 

291 

38.03 

323 

900 

8 

light  rain 

292 

34.13 

315 

1600 

3 

heavy  rain 

290 

32.20 

322 

900 

5 

Ught  rain 

292 

31.60 

314 

3500 

1 

Ught  snow 

291 

31.28 

324 

1699 

3 

heavy  snow 

288 

25.52 

319 

600 

8 

moderate  rain 

290 

24.17 

310 

800 

1 

none 

290 

23.17 

306 

900 

4 

heavy  rain 

288 

22.84 

308 

600 

3 

moderate  snow 

291 

16.74 

305 

2000 

1/2 

moderate  rain 

291 

15.87 

311 

4000 

1/2 

heavy  snow 

289 

12.49 

312 

600 

1 

moderate  snow 

289 

11.94 

304 

900 

1 

heavy  snow 

290 

11.24 

320 

3500 

3 

freezing  rain 

291 

7.39 

318 

1600 

1 

freezing  rain 

291 

4.39 

302 

600 

5 

freezing  rain 

291 

4.29 
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Table  12.  Overwater  Policy  Group  Comparison 


GROUP 

Total 

W-1 

W-2 

W-3 

W-4 

Number  of  Cases 

319 

24 

139 

72 

84 

Mean  Comfort  Level 

39.98 

13.79 

31.76 

46.73 

55.29 

BACKGROUND  VARIABLE 

Flying  History 

Total  Hours  Mean 

3845  hrs 

2711  hrs 

3174  hrs 

3950  hrs 

3771  hrs 

Last  90  Days  Mean 

36  hrs 

32  hrs 

38  hrs 

32  hrs 

38  hrs 

Mean  Age 

52 

56 

52 

51 

52 

Percent  Owning  Aircraft 

52 

67 

50 

50 

52 

Mean  Terrain  Experience 

Route  1  “  Overwater 

2.84 

2.29 

2.80 

3.17 

2.77 

Route  2  -  Flatland 

4.42 

4.29 

4.38 

4.46 

4.47 

Route  3  -  Mountain 

3.48 

3.70 

3.57 

3.30 

3.44 

Mean  Precipitation  Experience 

Light  Rain 

3.94 

3.65 

3.93 

4.04 

3.95 

Moderate  Rain 

3.36 

3.00 

3.31 

3.44 

3.46 

Heavy  Snow 

2.14 

1.83 

2..08 

2.43 

2.07 

Mean  Visibility  Experience 

INM 

3.15 

2.88 

3.03 

3.46 

3.18 

4NM 

4.18 

3.91 

4.14 

4.33 

4.20 

8NM 

4.72 

4.52 

4.72 

4.81 

4.70 

Mean  Ceiling  Experience 

800  ft. 

3.14 

2.92 

3.04 

3.39 

3.16 

1800  ft. 

4.05 

3.75 

3.98 

4.23 

4.09 

4000  ft. 

4.78 

4.71 

4.78 

4.81 

4.76 

Percent  Using  Weather  Source 

FAA/FSS 

58% 

57% 

54% 

57% 

66% 

DUAT 

26% 

32% 

26% 

27% 

25% 

TV/Radio 

41% 

36% 

43% 

38% 

40% 

Other 

16% 

12% 

20% 

13% 

14% 

Percent  Certificates  Held 

Private 

33% 

29% 

32% 

33% 

35% 

Commercial 

44% 

54% 

45% 

38% 

45% 

Airline  Transport 

20% 

13% 

21% 

26% 

15% 

Other 

3% 

4% 

2% 

3% 

5% 

Multi-Engine  Rating 

49% 

33% 

50% 

44% 

55% 

CFi/cni 

34% 

33% 

37% 

28% 

32% 

Instrument  Rating 

77% 

71% 

78% 

78% 

77% 

Percent  Flying  Job 

Employed  as  a  Pilot 

19% 

17% 

17% 

29% 

15% 

Mean  Number  of  FAA  Safety  Seminars 

1.69 

1.75 

1.40 

2.08 

1.81 

Mean  Lowest  Ceiling  for  Day  VFR  Trip 

2148  ft 

2775  ft 

2312  ft 

2093  ft 

1882  ft 

23 


Table  13.  Highlights  of  Group  Characteristics  (Overwater  Route) 


Highlights  c 

>f  Group  Characteristics  (Overwater  Route) 

Probability 

W-1 

W-2 

W-3 

W-4 

Compared  to  W-3,  W-2  has: 

0.00 

13.79 

31.76 

46.73 

55.29 

a  lower  comfort  level  for  overwater  route 

0.02 

17% 

17% 

29% 

15% 

a  lower  percentage  employed  as  a  pilot 

0.02 

1.75 

1.40 

2.08 

1.81 

less  attendance  at  FAA  Safety  Seminars 

0.04 

2.29 

2.80 

3.17 

2.77 

less  experience  flying  overwater 

0.04 

1.83 

2.08 

2.43 

2.07 

less  experience  flying  on  heavy  snow 

0.02 

2.88 

3.03 

3.46 

3.18 

less  experience  flying  in  1  NM  visibility 

0.05 

2.92 

3.04 

3.39 

3.16 

less  experience  flyng  in  800ft  ceilings 

0.05 

3.75 

3.98 

4.23 

4.09 

less  experience  flying  in  1800ft  ce 

!  lings 

1 

Probability  j  V/-1 

W-2 

W-3 

W-4 

Compared  to  W-1,  W-2  has: 

0.00  13.79 

31.76 

46.73 

55.29 

a  higher  comfort  level  for  overwater  route 

0.05 

2775ft 

2312ft 

2093ft 

1882fi 

a  lower  personal  ceiling  for  a  day  VFR  trip 

Probability 

W*1 

W-2 

W-3 

W-4 

Compared  to  W-4,  W-2  has: 

0.00 

13.79 

31.76 

46.73 

55.29 

a  lower  comfort  level  for  overwater  route 

0.00 

2775ft 

2312ft 

2093ft 

1882ft 

a  higher  personal  ceiling  for  a  day  VFR  trio 

0.01 

57% 

54% 

56% 

66% 

a  lower  percentage  obtain 

weather  from  FAA/FSS 

i 

1 

Probability 

W-t 

vy-2 

W-3 

W-4 

Compared  to  W-1,  W-3  has: 

0.00 

13.79 

31.76 

46.73 

55.29 

a  higher  comfort  level  for  overwater  route 

0.02 

2775ft 

2312-ft 

2(^3ft 

1882ft 

a  lower  personal  ceiling  for  a  day  VFR  trip 

0.01 

2.29 

2.80 

3.17 

2.77 

more  experience  flying  overwater 

0.05 

3.65 

3.93 

4.04 

3.95 

more  experience  flying  in  light  rain 

0.03 

1.83 

2.08 

2.43 

2.07 

more  experience  flying  in  heavy  snow 

0.02 

2.88 

3.03 

3.46 

3.18 

more  experience  flying  in  1  NM  visibility 

0.04 

3.91 

4.14 

4.33 

4.20 

more  experience  flying  in  4  NM  visibility 

0.03 

4.76 

4.69 

4.86 

4.38 

more  experience  flying  with  visibility  at  8NM 

0.03 

3.75 

3.98 

4.23 

4.09 

more  experience  flying 

in  1800ft  ceilings 

Probability 

W-1 

W-2 

W-3 

W-4 

Compared  to  W-4,  W-3  has: 

0.00 

13.79 

31.76 

46.73 

55.29 

a  lower  comfort  level  for  overwater  route 

0.05 

13% 

21% 

26% 

15% 

a  higher  percentage  of  ATP  rated  pilots 

0.02 

17% 

17% 

29% 

15% 

a  higher  percentage  employed  as  a  pilot 

0.04 

2.29 

2.80 

3.17 

2.77 

more  experience  flying  overwater  I 

0.05 

1.83 

2.08 

2.43 

2.07  1 

more  experience  flying 

in  heavy  snow 

r 

Probability 

W-1 

W-2 

W-3 

W-4  I 

Compared  to  W-4,  W-1  has: 

0.00 

13.79 

31.76 

46.73 

55.29  i 

a  lower  comfort  level  for  overwater  route 

0.03 

33% 

50% 

44% 

55%  i 

a  lower  percentage  of  multi-engine  rated  pilots 

0.00 

2775ft 

2312ft 

2093ft 

1882ft  i 

a  higher  personal  ceiling  for  a  day  VFR  trip 
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Table  14.  Non-Mountainous  Policy  Group  Comparison 


GROUP 

Total 

NM-1 

NM-2 

NM-3 

NM-4 

Number  of  Cases 

315 

49 

72 

93 

101 

Mean  Comfort  Level 

42.45 

20.85 

38.43 

43.82 

54.53 

BACKGROUND  VARIABLE 

Flying  History 

Total  Hours  Mean 

3835  hrs 

1849  hrs 

3950  hrs 

3372  hrs 

3955  hrs 

Last  90  Days  Mean 

35  hrs 

18  hrs 

37  hrs 

41  hrs 

37  hrs 

Mean  Age 

52  yrs 

53  yrs 

52  yrs 

50  yrs 

53  yrs 

Percent  Owning  Aircraft 

52% 

59% 

51% 

47% 

53% 

Mean  Terrain  Experience 

Route  1  “  Overwater 

2.82 

2.29 

2.93 

2.80 

3.01 

Route  2  -  Flatland 

4.41 

4.26 

4.26 

4.49 

4.53 

Route  3  ”  Mountain 

3.47 

3.43 

3.61 

3.27 

3.59 

Mean  Precipitation  Experience 

Light  Rain 

3.95 

3.65 

3.99 

4.01 

4.00 

Moderate  Rain 

3.36 

3.00 

3.32 

3.44 

3.49 

Heavy  Snow 

2.13 

1.83 

1.96 

2.21 

2.32 

Mean  Visibility  Experience 

INM 

3.15 

2.83 

3.17 

3.18 

3.26 

4NM 

4.18 

3.67 

4.14 

4.34 

4.31 

8NM 

4.72 

4.38 

4.69 

4.76 

4.86 

Mean  Ceiling  Experience 

800  ft. 

3.13 

2.73 

3.18 

3.17 

3.25 

1800  ft. 

4.05 

3.63 

4.10 

4.10 

4.16 

4000  ft. 

4.77 

4.52 

4.74 

4.85 

4.86 

Percent  Using  Weather  Source 

PAA/ESS 

58% 

56% 

53% 

61% 

60% 

DUAT 

26% 

32% 

28% 

26% 

23% 

TV/Radio 

41% 

50% 

34% 

45% 

37% 

Other 

16% 

17% 

14% 

18% 

13% 

Percent  Certificates  Held 

Private 

33% 

43% 

33% 

30% 

30% 

Commercial 

45% 

39% 

38% 

47% 

50% 

Airline  Transport 

19% 

10% 

28% 

19% 

18% 

Other 

3% 

8% 

1% 

3% 

2% 

Multi-Engine  Rating 

49% 

33% 

51% 

49% 

53% 

CFI/CFII 

34% 

27% 

33% 

34% 

37% 

Instrument  Rating 

77% 

67% 

81% 

83% 

73% 

Percent  Flying  Job 

Employed  as  a  Pilot 

19% 

6% 

21% 

24% 

19% 

Mean  Number  of  FAA  Safety  Seminars 

1.74 

1.49 

2.11 

1.66 

1.66 

Mean  Lowest  Ceiling  for  Day  VFR  Trip 

2174  ft 

2718  ft 

2490  ft 

1979  ft 

1863  ft 
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Table  15.  Highlights  of  Group  Characteristics  (Non-Mountainous) 


Hiahliahts  of  Group  Characteristics  (Non-Mountainous  Route) 

Probabiiitv 

NM-1 

NM-2 

NM-3 

NM-4 

Comoared 

to  NM-4.  r 

JM-2  has: 

0.00 

20.85 

38.43 

43.82 

54.53 

a  lower  comfort  level 

for  non-mountalnous  route 

0.00 

2718ft 

2490ft 

1979ft 

1836ft 

a  hlaher  personal  ceilina  for  a  dav  VFR  trio 

0.02 

4.26 

4.26 

4.49 

4,53 

less  experience  over  non-mountainous  terrain 

0.04 

1.83 

1.96 

2.21 

2.32 

less  experience  fivinq  In  heavv  snow 

0.02 

4.38 

4.69 

4.76 

4.86 

less  experience  fivinq  with  visibilitv  at  8NM 

0.05 

39% 

38% 

47% 

50% 

a  lower  percentaqe  of  commercial  pilots 

Probabiiitv 

NM-1 

NM-2 

NM-3 

NM-4 

Compared  to  NM-3.  NM-2  has: 

0.00 

20.85 

38.43 

43.82 

54.53 

a  lower  comfort  level 

for  non-mountainous  route 

0.00 

2718ft 

2490ft 

1979ft 

1863ft 

a  hlqher  personal  ceilinq  for  a  dav  VFR  trip 

0.03 

50% 

34% 

45% 

37% 

a  lower  percentage  obtain  weather 

from  TV/Radio  sourcei 

) 

Probabiiitv 

NM-1 

NM-2 

NM-3 

NM-4 

Compared  to  NM-1.  NM-2  has: 

0.00 

20.85 

38.43 

43.82 

54.53 

a  hiqher  comfort  level 

for  non-mountainous  route 

0.01 

18% 

10% 

19% 

28% 

a  lower  percentage  of  ATP  rated  pilots 

0.05 

67% 

81% 

83% 

73% 

a  hiqher  percentaqe  of  instrument  rated  pilots 

0.02 

33% 

51% 

49% 

53% 

a  higher  percentage  of  multi-engine  rated  pilots 

0.01 

6% 

21% 

24% 

19% 

a  higher  percentage  employed  as  a  pilot 

0.01 

18hrs 

37hrs 

41hrs 

37hrs 

more  fivinq  hours  in  the  last  90  davs 

0.01 

1849hrs 

3950hrs 

3372hrs 

3955hrs 

a  hiqher  number  of  total  fivinq  hours 

0.01 

50% 

34% 

45% 

37% 

a  lower  percentaqe  obtain  weather 

from  TV/Radio  sources 

0.01 

2.29 

2.93 

2.80 

3.01 

more  experience  fivinq  overwater 

0.04 

3.65 

3.99 

4.01 

4.00 

more  experience  fivinq  in  light  rain 

0.01 

3.67 

4.14 

4.34 

4.31 

more  experience  flying  with  4NM  visibilitv 

0.02 

4.38 

4.69 

4.76 

4.86 

more  experience  fivinq  with  8NI\/l  visibilitv 

0.04 

2.73 

3.18 

3.17 

3.25 

more  experience  fivinq  in  800ft  ceilings 

0.01 

3.62 

4.10 

4.10 

4.16 

more  experience  fivinq 

in  1800ft  ceilings 

Probabiiitv 

NM-1 

NM-2 

NM-3 

NM-4 

Compared  to  NM-3,  NM-4  has: 

0.00 

20.85 

38.43 

43.82 

54.53 

a  hiqher  comfort  level 

for  non-mountainous  route 

0.05 

3.43 

3.61 

3.27 

3.59 

more  experience  over  mountainous  terrain 
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Table  15.  Highlights  of  Group  Characteristics  (Non-Mountainous) 

(Continued) 


Probability 

NM-1 

NM-2 

NM-3 

NM-4 

Compared  to  NM-I ,  NM-4  has: 

0.00 

20.85 

38.43 

43.82 

54.53 

a  higher  comfort  level 

for  non-mountalnous  route  1 

0.03 

39% 

38% 

47% 

50% 

a  higher  percentage  of  commercial  pilots 

0.01 

33% 

51% 

49% 

53% 

a  higher  percentage  of  multi-engine  rated  pilots 

0.02 

6% 

21% 

24% 

19% 

a  higher  percentage  employed  as  a  pilot 

0.00 

2718ft 

2490ft 

1979ft 

1863ft 

a  lower  personal  ceiling  for  a  dav  VFR  trip 

0.01 

18hrs 

37hrs 

41hrs 

37hrs 

more  fivino  hours  in  the  last  90  davs 

0.00 

1849hrs 

3950hrs 

3372hrs 

3955hrs 

a  higher  number  of  total  flying  hours 

0.02 

50% 

34% 

45% 

37% 

a  lower  percentage  obtain  weather 
from  TV/Radio  sources 

0.00 

2.29 

2.93 

2.80 

3.01 

more  experience  flying  overwater 

0.03 

4.26 

4.26 

4.49 

4.53 

more  experience  over  non-mountainous  terrain 

0.02 

3.65 

3.99 

4.01 

4.00 

more  experience  flving  in  light  rain 

0.01 

3.00 

3.32 

3.44 

3.49 

more  experience  flving  in  moderate  rain 

0.02 

1.83 

1.96 

2.21 

2.32 

more  experience  flving  In  heavy  snow 

0.03 

2.83 

3.17 

3.18 

3.26 

more  experience  fivino  with  1NM  visibilitv 

0.00 

3.67 

4.14 

4.34 

4.31 

more  experience  fivino  with  4NM  visibilitv 

0.00 

4.38 

4.69 

4.76 

4.86 

more  experience  flying  with  8NM  visibility 

0.01 

2.73 

3.18 

3.17 

3.25 

more  experience  flving  in  800ft  ceilings 

0.00 

3.62 

4.10 

4.10 

4.16 

more  experience  fivino  in  1800ft  ceilings 

0.00 

4.52 

4.74 

4.85 

4.86 

more  experience  flying 

in  4000ft  ceilings 

Probability 

NM-1 

NM>2 

NM-3 

NM-4 

Compared  to  NM-1.  NM-3  has: 

0.00 

20.85  I 

38.43 

43.82 

54.53 

a  higher  comfort  level 

_ 

for  non-mountainous  route  1 

0.02 

67% 

81% 

83% 

73% 

a  higher  percentage  of  instrument  rated  pilots 

0.03 

33% 

51% 

49% 

53% 

a  higher  percentage  of  multi-engine  rated  pilots 

0.00 

6% 

21% 

24% 

19% 

a  higher  percentage  employed  as  a  pilot 

0.00 

271 8fl 

2490ft 

1979ft 

1863ft 

a  lower  personal  ceiling  for  a  dav  VFR  trio 

0,01 

18hrs 

37hrs 

41hrs 

37hrs 

more  flving  hours  in  the  last  90  davs 

0.02 

1849hrs 

3950hrs 

3372hrs 

3955hrs 

a  higher  number  of  total  flying  hours 

0.02 

2.29 

2.93 

2.80 

3.01 

more  experience  flving  overwater 

0.03 

3.65 

3.99  ! 

4.01 

4.00 

more  experience  flving  in  light  rain 

0.03 

3.00 

3.32  1 

3.44 

3.49 

more  experience  flving  in  moderate  rain 

0.04 

1.83 

1.96 

2.21 

2.32 

more  experience  flving  in  heavy  snow 

0.00 

3.67 

4.14 

4.34 

4.31 

more  experience  flving  with  4NM  visibilitv 

0.00 

4.38 

4.69 

4.76 

4.86 

more  experience  flving  with  8NM  visibilitv 

0.04 

2.73 

3.18 

3.17 

3.25 

more  experience  flving  in  800ft  ceilings 

0.01 

3.62 

4.10 

4.10 

4.16 

more  experience  flving  in  1800ft  ceilings 

0.00 

4.52 

4.74 

4.85 

4.86 

more  experience  fivino  in  4000ft  ceilings 

27 


Table  16.  Mountainous  Policy  Group  Comparisons 


GROUP 

Total 

M-1 

M-2 

M-3 

M-4 

Number  of  Cases 

303 

94 

78 

63 

68 

Mean  Comfort  Level 

36.38 

31.61 

35.57 

40.71 

53.72 

BACKGROUND  VARIABLE 

Flying  History 

Total  Hours  Mean 

3775  hrs 

3013  hrs 

3605  hrs 

3246  hrs 

3801  hrs 

Last  90  Days  Mean 

36hrs 

30  hrs 

47  hrs 

33  hrs 

32  hrs 

Mean  Age 

52  yrs 

53  yrs 

51  yrs 

51  yrs 

53  yrs 

Percent  Owning  Aircraft 

52% 

56% 

60% 

46% 

43% 

Mean  Terrain  Experience 

Route  1  -  Overwater 

2.81 

2.68 

2.94 

2.97 

2.68 

Route  2  -  Flatland 

4.41 

4.40 

4.45 

4.45 

4.36 

Route  3  -  Mountain 

3.46 

3.46 

3.46 

3.37 

3.55 

Mean  Precipitation  Experience 

Light  Rain 

3.94 

3.83 

4.04 

3.97 

3.95 

Moderate  Rain 

3.35 

3.24 

3.39 

3.35 

3.47 

Heavy  Snow 

2.12 

2.09 

2.01 

2.06 

2.32 

Mean  Visibility  Experience 

INM 

3.13 

2.94 

3.20 

3.34 

3.14 

4NM 

4.19 

3.99 

4.22 

4.32 

4.29 

8NM 

4.72 

4.58 

4.68 

4.86 

4.83 

Mean  Ceiling  Experience 

800  ft. 

3.11 

2.99 

3.20 

3.11 

3.18 

1800ft. 

4.04 

3.96 

1.01 

4.03 

4.20 

4000  ft. 

4.78 

4.70 

4.74 

4.87 

4.83 

Percent  Using  Weather  Source 

FAA/FSS 

58% 

54% 

61% 

53% 

62% 

DUAT 

26% 

31% 

29% 

24% 

19% 

TV/Radio 

40% 

45% 

39% 

42% 

32% 

Other 

15% 

16% 

13% 

18% 

13% 

Percent  Certificates  Held 

Private 

33% 

37% 

27% 

37% 

31% 

Commercial 

45% 

43% 

50% 

44% 

41% 

Airline  Transport 

19% 

19% 

18% 

19% 

22% 

Other 

3% 

1% 

5% 

0% 

6% 

Multi-Engine  Rating 

49% 

41% 

47% 

49% 

59% 

CFI/CFII 

34% 

29% 

38% 

33% 

38% 

Instrument  Rating 

77% 

76 

83% 

79% 

69% 

Percent  Flying  Job 

Employed  as  a  Pilot 

19% 

14% 

22% 

21% 

21% 

Mean  Number  of  FAA  Safety  Seminars 

1.69 

1.62 

1.9 

1.86 

1.41 

Mean  Lowest  Ceiling  for  Day  VFR  Trip 

2167  ft. 

2323  ft. 

2164  ft 

2051  ft. 

2063  ft. 
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Table  17.  Highlights  of  Group  Characteristics  (Mountainous) 


Highlights  of  Group  Characteristics  (Mountainous  Route) 

Probability 

M-1 

M-2 

M-3 

M-4 

Compared 

to  M-2,  M-1  has: 

0.00 

21.61 

35.57 

40.71 

53.72 

a  lower  comfort  level  for  mountainous  route 

0.03 

30hrs 

47hrs 

33hrs 

32hrs 

less  flying  hours  in  the  last  90  days 

Probability 

M-1 

M-2 

M-3 

M-4 

Compared  to  M-4,  M-1  has: 

0.00 

21.61 

35.57 

40.71 

53.72 

a  lower  comfort  level  for  mountainous  route 

0.01 

41% 

47% 

49% 

59% 

a  lower  percentage  of  multi-engine  rated  pilots 

0.02 

76% 

83% 

79% 

69% 

a  higher  percentage  of  instrument  rated  pilots 

0.02 

56% 

60% 

46% 

43% 

a  higher  percentage  own  their  ovm  airplane 

0.02 

31% 

29% 

24% 

19% 

a  higher  percentage  obtain  weather  from  DUAT 

0.02 

45% 

39% 

42% 

32% 

a  higher  percentage  obtain 

weather  from  TV/Radio  sources 

0.03 

3.99 

4.22 

4.32 

4.29 

less  experience  flying  with  4NM  visibiiity 

0.01 

4.58 

4.68 

4.86 

4,83 

less  experience  flying  with  8NM  visibility 

Probability 

M-1 

M-2 

M-3 

M-4 

Compared  to  M-3,  M-1  has: 

0.00 

21.61 

35.57 

40.71 

53.72 

a  lower  comfort  level  for  mountainous  route 

0.04 

2.94 

3.20 

3.34 

3.14 

less  experience  flying  with  1 NM  visibility 

0.02 

3.99 

4.22 

4.32 

4.29 

less  experience  flying  with  4NM  visibility 

0.01 

4.58 

4.68 

4.86 

4.83 

less  experience  flying  with  8NM  visibility 

1 

Probability 

M-1 

M-2 

M-3 

M-4 

Compared  to  M-4,  M-2  has: 

0.00 

21.61 

35.57 

40.71 

53.72 

a  lower  comfort  level  for  mountainous  route 

0.02 

76% 

83% 

79% 

69% 

a  higher  percentage  of  instrument  rated  pilots 

0.02 

56% 

60% 

46% 

43% 

a  higher  percentage  own  their  own  airplane 

0.05 

4.58 

4.68 

4.86 

4.83 

less  experience  flying  with  8NM  vis 

sibility 

Probability 

M-1 

M-2 

M^3 

M-4 

Compared  to  M-3,  M-2  has; 

0.00 

21.61 

35.57 

40.71 

53.72 

a  lower  comfort  level  for  mountainous  route 

0.05 

56% 

60% 

46% 

43% 

a  higher  percentage  own  their  own  airplane 

0.03 

4.58 

4.68 

4.86 

4.83 

less  experience  flying  with  8NM  visibility 

0.05 

4.70 

4.74 

4.87 

4.83 

less  experience  flying  in  4000ft  ceilings 

1  1 

Probability 

M-1 

M-2 

M-3 

M-4 

Compared  to  M-3,  M-4  has: 

0.00 

21.61 

35.57 

40.71 

53.72 

a  higher  comfort  level  for  mountainous  route 

Table  18.  Mean  Regional  Comfort  Ratings  by  Terrain 

Terrain  Type 


Redon 

Water 

Alaska 

42.69 

Northwest 

36.23 

Western 

41.95 

Southern 

42.76 

New  England 

39.48 

Eastern 

38.99 

Total  Sample 

39.98 

Non-Mountainous 

Mountainous 

56.43 

48.23 

38.14 

34.29 

45.09 

37.09 

45.49 

37.50 

40.08 

35.25 

40.07 

34.44 

42.45 

36.38 
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Table  19.  Mean  Regression  Weights  for  Geographical  Regions  by  Terrain 


Over  Water 


Variable 

AK 

NW 

WP 

SO 

NE 

EA 

Average 

0.69 

0.68 

0.63 

0.62 

0.67 

0.70 

Ceiling  (A) 

-0.44 

Visibility  (B) 

Precipitation  (C  ) 

-0.63 

-0.33 

-0.25 

-0.38 

-0.33 

A*B*C 

0.65 

1.60 

1.30 

1.17 

1.75 

1.60 

A*B 

-0.26 

-0.43 

-0.34 

-0.23 

-0.89 

-0.67 

A*C 

0.79 

B*C 

0.27 

-0.22 

-0.25 

Non-Mountainous 

Variable 

AK 

NW 

WP 

SO 

NE 

EA 

Average 

0.82 

0.73 

0.75 

0.72 

0.71 

0.76 

Ceiling  (A) 

-0.29 

-0.47 

-0.43 

-0.41 

-0.33 

-0.52 

Visibility  (B) 
Precipitation  (C ) 
A*B*C 

0.23 

0.25 

A*B 

0.57 

0.70 

0.85 

0.72 

0.72 

0.82 

A*C 

0.31 

0.22 

0.32 

0.32 

0.21 

B*C 

-0.25 

-0.23 

-0.43 

-0.33 

-0.40 

Mountainous 

Variable 

AK 

NW 

WP 

SO 

m 

East 

Average  R^ 

0.79 

0.76 

0.79 

0.76 

0.74 

0.78 

Ceiling  (A) 

Visibility  (B) 

-0.39 

-0.32 

-0.49 

-0.39 

-0.32 

-0.42 

Precipitation  (C ) 
A*B*C 

0.47 

1.00 

0.74 

0.91 

1.03 

1.34 

A*B 

0.38 

0.22 

0.32 

0.32 

0.21 

A*C 

0.31 

0.40 

B*C 

-0.25 

-0.23 

-0.43 

-0.33 

-0.40 
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Table  20.  I^oporfionality  of  Group  Membership  by  Re^on 


Itotal  cases 


Alaskan  Region  _ 

Southern  Region _ 

New  England  Region _ 

Western  Pacific  Region _ 

Northwest  Mountains  Region 
Eastern  Region _ 


319 

TotalW 

12 

50 

82 

49 

56 

57 


24 

W-1 

2 

2 

7 

5 

5 

2 


Itotal  cases 


Alaskan  Region _ 

Southern  Region _ 

New  England  Region _ 

Western  Pacific  Region _ 

Northwest  Mountains  Region 
Eastern  Region _ 


315 

TotalNM 

13 

50 

81 

48 

57 

55 


49 

NM-1 

1 

5 

19** 

5 
12 

6 


TOTAL  CASES 


Alaskan  Region _ 

Southern  Region _ 

New  England  Region _ 

Western  Pacific  Region 
Northwest  Mountains  Region 
Eastern  Region _ 


315 

TotalM 

13 

48 

79 

47 

54 

52 


49 

M-1 

4 

15 

27 

14 

15 
18 


Legend  I  I 

*  =  Statistically  significant  t-test.  (.05)  below  the  mean 
**  =  Statistically  significant  t-test.  (.05)  above  the  mean 


OVEfiWATER  ROUTE 


139 


W-2 


72 


W-3 


84 


W-4 


23 


16 


36 


22 


17 


19 


11 


14 


33 


22 


16 


17 


NON-MOUNTAINOUS  ROUTE 


72 


NM-2 


93 


NM-3 


101 


NM-4 


10* 


10 

18 


14 


21 


24 


20 


10 


14 


19 


19 


15 


13 


21 


11** 

15 


MOUNTAINOUS  ROUTE 


72 

M-2 

1 

14 

22 

10 

17 

11 


93 

M-3 

1 

10 

15 

11 

9 

13 


101 

M-4 

7** 

9 
15 
12 
13 

10 


Table  21.  Comparison  of  Total  Sample  Demographic  Data 
with  South  Texas  Sample 

Variable  Current  Sample  South  Texas 


N 

326 

150 

Average  Age 

52  years 

42 

Mean  Total  Hying  Hours 

3845 

1694 

Mean  Comfort  Level 

Mean  Over-Water  Comfort  Level 

39.80 

30.29 

Mean  Non-mountain  Comfort  Level 

42.27 

30.15 

Mean  Mountain  Comfort  Level 

36.38 

30.29 
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Table  22.  Regional  and  South  Texas  Demographics 


i  ROUTE  1  OVERW^ER 

i  MEAN  COMFORT  LEVEL  ‘ 

13.79 

27 

31.76  j 

40 

46.73 

55.29 

i  number  of  CASES  (31^150) 

24 

112 

139 

38 

72 

84 

i  BACKGROUND  VARIABLE 

W-1 

GP  A 

W-2 

QP  g 

Vy.^ 

i  Flying  History 

Total  Hours 

2711hrs 

1896hrs 

3174hrs 

1187hts 

3950hrs 

3771  hrs 

Last  90  Days 

32hrs 

39hrs 

38hrs 

39hrs 

32hrs 

38hrs 

lli/iean  Age 

56  yrs 

43yrs 

52  yrs 

40yrs 

51  yrs 

52  yrs 

[Percent  Owing  Aircraft 

67% 

39% 

50% 

26% 

50% 

52% 

[Percent  Female 

n/a 

12% 

n/a 

16% 

n/a 

n/a 

[Mean  Terrain  Experience 

Route  1  -  0\erwater 

2.29 

1.04 

2.80 

0.89 

3.17 

2.77 

Route  2  -  Flatland 

4.29 

n/a 

4.38 

n/a 

4.46 

4.47 

Route  3  -  Mountainous 

3.70 

n/a 

3.57 

n/a 

3.30 

3.44 

[Mean  Precipitation  Experience 

Light  Rain 

3.65 

2.49 

3.93 

2.18 

4.04 

3.95 

Moderate  Rain 

I  3.00 

1.71 

3.31 

1.29 

3.44  i 

3.46 

Heavy  Snow 

1  1.1^ 

0.43 

2.08 

1  0.37 

2.43  1 

2 -- 

IMean  Visibility  Experience 

1 

i  1 

I  1  NM 

2.88 

1.74 

3.03 

1.42 

3.46 

I  3.18  1 

. ! 

i  4  NM 

3.91 

2.80 

4.14  i 

2.50 

4.33 

4.20  ! 

I  8  NM 

4.52 

3.62 

4.72  1 

3.37 

4.81 

4.70  1 

[Mean  Ceiling  Experience 

:  800  ft 

2.92 

1.77 

3.04 

1.50 

3.39 

3.16 

1800  ft 

3.75 

1  2.94 

3.98 

2.68 

4.23 

4.09 

i  4000  ft 

4.71 

1  3.65 

4.78 

3.53 

. '"4.81 _ 

4.76 

Percent  Using  Weat^r  Source 

\ 

FAA/FSS 

57% 

\  67% 

54% 

65% 

57% 

[  66% 

DUAT 

i  32% 

n/a 

26% 

n/a 

27% 

25% 

TV/Radio 

36% 

i  33%  . 

43% 

24% 

38% 

40% 

Other 

12% 

^'12%'''''' 

20% 

'"~  _7% 

13% 

14% 

Percent  Certificates  Held 

1 

PP 

29% 

1  58% 

32% 

63°/o 

33% 

35% 

COM 

54% 

f"~33%~ . 

45%”'“ 

21% 

38% 

45% 

ATP 

13% 

i  n/a 

21% 

n/a 

26% 

15% 

Other 

4% 

I  n/a 

2% 

n/a 

3% 

1  5% . 

Muiti-Engine  Rating 

33% 

n/a 

50% 

n/a 

44% 

55% 

CFI/CFIi 

33% 

21% 

37% 

18% 

28% 

32% 

Instrument  Rating 

T . 

*‘"46%'""“ 

78% 

39% 

78% 

'"''”77%""” 

Percent  Flying 

Job-empioyed  as  a  pilot 

17% 

;  14% 

17% 

19% 

29% 

15% 

[Average  Number 

1.75 

n/a 

1.40 

n/a 

2l)8 

1.81 

i  FAA  Safety  Seminars  Attended 

[  Lowest  Ceiling  for  Day  VFR  trip 

2775ft 

n/a 

2312ft 

n/a 

'209M 

i  882ft 

[Average  R^ 

0.69 

0.67 
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Table  23.  Regional  and  South  Texas  Demographics 
ROUTE  2  NON-MOUNT AINOUS  I  \  ^  i  ^  T 


MEAN  COMFORT  LEVEL 

20.85 

17 

38.43 

28 

43.82 

32 

54.53 

50 

NUMBER  OF  CASES  (315/150) 

49 

37 

72 

52 

93 

34 

101 

27 

BACKGROUND  VARIABLE 

NM-1 

GP4 

NM-2 

GP2 

NM-3 

GP1 

NM-4 

GP3 

Flying  History 

Total  Hours 

1849hrs 

\1679hrs  3950hrs 

\l516hrs  3372hrs 

\1942hrs  Z9S5hrs 

1869hrs 

Last  90  Days 

18hrs 

25hrs 

37hrs 

37hrs 

41hrs 

51hrs 

37hrs 

47hrs 

Average  Age 

53yrs 

46yrs 

52yrs 

41yrs 

50yrs 

44yrs 

53yrs 

38yrs 

Percent  Owing  Aircraft 

59% 

41% 

51% 

44% 

47% 

32%, 

53% 

19%, 

Percent  Female 

n/a 

14% 

n/a 

17% 

n/a 

12%, 

n/a 

4%o 

Mean  Terrain  Experience 

Route  1  -  Overwater 

2.29 

n/a 

2.93 

n/a 

2.80 

n/a 

3.01 

n/a 

Route  2  -  Flatland 

4.26 

3.49 

4.26 

3.65 

4.49 

3.50 

4.53 

3.41 

Route  3  -  Mountainous 

3.43 

n/a 

3.61 

n/a 

3.27 

n/a 

3.59 

n/a 

Mean  Precipitation  Experience 

Light  Rain 

3.65 

2.11 

3.99 

2.50 

4.01 

2.62 

4.00 

2.41 

Moderate  Rain 

3.00 

1.27 

3.32 

1.71 

3.44 

1.79 

3.49 

1.63 

Heavy  Snow 

1.83 

0.38 

1.96 

0.33 

2.21 

0.38 

2.32 

0.67 

Mean  Visibility  Experience 

1  NM 

2.83 

1.38 

3.17 

1.79 

3.18 

1.62 

3.26 

r  1.85 

4NM 

3.67 

2.24 

4.14 

2.92 

4.34 

2.74 

4.31 

3.00 

8NM 

4.38 

3.30 

4.69 

3.77 

4.76 

3.38 

4.86 

3.70 

Mean  Ceiling  Experience 

800  ft 

2.73 

1.62 

3.18 

1.75 

3.17 

1.65 

3.25 

1.78 

1800  ft 

3.63 

2.46 

4.10 

3.13 

4.10 

2.88 

4.16 

2.93 

4000  ft 

4.52 

3.38 

4.74 

3.81 

4.85 

3.56 

4.86 

3.67 

Percent  Using  Weather  Source 

FAA/FSS 

56% 

68% 

53% 

66% 

61% 

74% 

60% 

56% 

DUAT 

32% 

n/a 

28% 

n/a 

26% 

n/a 

23% 

n/a 

TV/Radio 

50% 

26% 

34% 

32% 

45% 

33%, 

37% 

35% 

Other 

17% 

13% 

14% 

12% 

18% 

10%, 

13% 

7%o 

Percent  Certificates  Held 

PP 

43% 

70% 

33% 

58% 

30% 

53%, 

30% 

56%, 

COM 

39% 

24% 

38% 

37% 

47% 

32%, 

50% 

22%o 

ATP 

10% 

n/a 

28% 

n/a 

19% 

n/a 

18% 

n/a 

Other 

8% 

n/a 

1% 

n/a 

3% 

n/a 

2% 

n/a 

Multi-Engine  Rating 

33% 

n/a 

51% 

n/a 

49% 

n/a 

53% 

n/a 

CFI/CFII 

27% 

16% 

33% 

21% 

34% 

21% 

37% 

26% 

Instrument  Rating 

67% 

n/a 

81% 

38% 

83% 

n/a 

73% 

n/a 

Percent  Flying 

Job-employed  as  a  pilot 

6% 

3% 

21% 

17% 

24% 

21% 

19% 

23% 

Average  Number 

1.49 

n/a 

2.11 

n/a 

1.66 

n/a 

1.66 

n/a 

FAA  Safety  Seminars  Attended 

Lowest  Ceiling  for  Day  VFR  Trip 

2718ft 

n/a 

2490ft 

n/a 

n/a 

1863ft 

n/a 

Average  R* 

0.58  1 

1 

0.77  1 

1979ft  ^ 

0.79  1 

0.50 
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Table  24.  Regional  and  South  Texas  Demographics 


ROUtE  3  MOUt^AiNOUS 

\ 

i 

! 

_ 4- 

I 

\ 

i 

1 

21.61 

17  I 

\  35.57 

29  I 

40.71 

36  j 

S3.72 

46 

{NUMBER  OF  CASES  ^03/1^ 

36  \ 

78 

is 

14  1 

68  1 

31  1 

j 

BACKGROUND  VARIABLE 

M-1 

GP4 

M-2 

QP2 

M-3 

Qpy 

M-4 

GP  3  i 

Flying  History 

Total  Hours 

3013hrs 

2140hrs 

3605hrs 

1963hrs 

3246hrs 

1969hrs 

^Olhrs 

5^h!S\ 

Last  90  Days 

30hrs 

21hrs 

47hrs 

49hrs 

33hrs 

47hrs 

32hrs 

34hrs  i 

Average  Age 

53  yrs 

48yrs 

51  yrs 

41  yrs 

51  yrs 

51  yrs 

53  yrs 

34yrs  i 

Percent  Owing  Aircraft 

56% 

44% 

60% 

39% 

46% 

'43% 

43% 

16%,  ’ 

Percent  Femaie 

n/a 

25% 

n/a 

10% 

n/a 

7% 

n/a 

6%  i 

Mean  Terrain  Experience 

^  Route  1  -  Ovenwater 

2.68 

n/a 

2.94 

n/a 

2.97 

n/a 

2.68 

n/a  1 

Route  2  -  Flatland 

4.40 

n/a 

4.45 

n/a 

4.45 

n/a 

4.36 

n/a  1 

Route  3  -  Mountainous 

3,46  ^ 

1.36 

3.46 

1.58 

3.37 

1.57 

3,55 

1.10  i 

{Mean  Precipitation  Experience 

Light  Rain 

3.83 

2.31 

4.04 

2.59 

3.97 

2  50 

^95 

2.10  i 

Moderate  Rain 

3.24 

1.42 

3.39 

1.81 

3.35 

2  00 

3.47 

1.19 

Heavy  Snow 

2.09 

0.58 

2.01 

0.43 

i06 

0  36 

2.32 

0.19  i 

[Mean  Visibility  Experience 

1  4  NM 

2.94 

1.39 

3.20 

1.90 

3.34 

1.66 

3.14 

1.35  1 

I  1  NM 

3.99 

2.25 

4.22 

2.94 

4.32 

3.14 

4.29 

2.61  1 

i  8NM 

I  4.58 

3.28 

4.68 

3.72 

4.86 

\  . 

4.83 

3.55  1 

i  Mean  Ceiiing  Experience 

:  800  ft 

t . 2^99 . ^ 

1.58 

3.20 

1.84 

3.11 

1  2.43 

118 

1  1.19 

i  1800  ft 

I  3.96 

2.50 

4.01 

3.10 

4^03  “ 

4.20 

,  2.52 

i  4000  ft 

i  4.70 

3.28 

4.74 

1  3.77 

4.87 

\  3.86 

4.83 

1  3.58 

i Percent  Using  Weather  Source 

i  FAA/FSS 

54% 

64% 

61% 

1  68°/o 

53% 

f  58% 

62% 

:  69% 

i  DUAT 

i  31% 

n/a 

29% 

\  n/a 

24% 

n/a 

19% 

j  n/a  i 

TV/Radio 

1  45% 

35% 

39% 

1  32% 

42% 

24% 

32% 

^  28%> 

Other 

i  16% 

17% 

13% 

1  1% 

18% 

\  7% 

13% 

5% 

Percent  Certificates  Held 

. 

PP 

37% 

61% 

27% 

'^'"'37%"'’ 

\  36% 

“66%“'“' 

COM 

43% 

33% 

50®/o 

33% 

""'■'44%' 

{  36% 

“““41% . 

16%o 

I  ATP 

19% 

n/a 

18% 

n/a 

19% 

n/a 

22% 

ri/a 

i  other 

ri% 

~n/a 

"“5%"~ 

n/a 

0% 

n/a 

6% 

n/a 

Multi-Engine  Rating 

41% 

n/a 

47% 

n/a 

49% 

n/a 

59%“ 

rt/a 

i  CFI/CFII 

— 2^^- 

s»% 

22% 

33% 

{  29% 

38% 

16% 

Instalment  Rating 

I  76% 

n/a 

83% 

I  36% 

79% 

n/a 

69% 

50% 

{Percent  Flying 

Job-employed  as  a  pilot 

!  14% 

5% 

22% 

I  22% 

21% 

"  76%  ^  ' 

■'“'21% . 

12% 

{Average  Number 

1.62 

n/a 

1.90 

n/a 

1.86 

n/a 

l41 

n/a 

I  FAA  Safety  Seminars  Attended 

[Lowe^  Ceiling  for  Day  VFR  trip 

2323ft 

n/a 

2164ft 

\  n/a 

2051ft 

n/a 

2063ft 

n/a 

{Average  R* 

0.60 

I  0.80 

0.83 

0.75 
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Table  25.  Compton  of  CompensatDij  and  Non-Compoisatoiy  Models 
Model  Corrdalion 


Over-water  Non-Mountainous  Mountainous 

^Tex.  Regions  S.  Texas  Regions  S.Tex.  Redons 

Compensatory  Models 


A+B+C  (Additive) 

0.83 

0.80 

0.94 

0.92 

0.93 

0.92 

A*B*C  (Multiplicative) 

0.89 

0.86 

0.96 

0.94 

0.96 

0.95 

A*B  (Multiplicative) 

0.65 

0.62 

0.86 

0.89 

0.84 

0.85 

0.83 

A*C  (Multiplicative) 

0.79 

0.74 

0.76 

0.73 

0.82 

B*C  (Multiplicative) 

0.73 

0.72 

0.79 

0.76 

0.86 

0.83 

Worst  Factor  Models 

Worst  Factor  (X>5.0) 

0.54 

0.59 

0.69 

0.74 

0.71 

0.73 

Worst  Factor  (X>4.5) 

0.78 

0.85 

0.80 

0.85 

0.82 

0.86 

Worst  Factor  (X>4.0) 

0.74 

0.83 

0.76 

0.82 

0.79 

0.83 

Non-Compensatorv  Models 

A 

0.46 

0.43 

0.48 

0.50 

0.53 

0.56 

0.60 

A  (X>5.0) 

0.46 

0.44 

0.44 

0.47 

0.55 

A(X>4.5) 

0.35 

0.41 

0.38 

0.45 

0.42 

0.48 

0.48 

A(X>4.0) 

0.35 

0.41 

0.38 

0.45 

0.42 

B 

0.48 

0.50 

0.67 

0.69 

0.62 

0.59 

B  (X>5.0) 

0.41 

0.43 

0.56 

0.59 

0.53 

0.50 

0.45 

B(X>4.5) 

0.46 

0.53 

0.62 

0.66 

0.46 

B  (X>4.0) 

0.46 

0.53 

0.62 

0.66 

0.46 

0.45 

C 

0.62 

0.58 

0.60 

0.52 

0.73 

0.72 

C  (X>5.0) 

0.57 

0.53 

0.60 

0.57 

0.67 

0.68 

C(X>4.5) 

0.50 

0.49 

0.53 

0.52 

0.56 

0J8 

C(X>4.0) 

0.41 

0.42 

0.35 

0.30 

0.56 

0.57 
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APPENDIX  A 


Software  and  Analysis 

Data  analysis  for  this  project  was  accomplished  on  a  Unisys  1 1 00  series  mainframe  computer  at  the  Human 
Resources  Directorate  of  the  USAF  Armstrong  Laboratoiy  (AL/HR)  and  on  the  IBM  RS-6000  RISC  model  530 
machine  at  the  Metrica  facility.  All  software  used  was  developed  by  and  is  the  property  of  the  USAF  Armstrong 
Laboratoiy,  Brooks  AFB,  San  Antonio,  Texas.  More  specifically,  in  addition  to  utility  and  custom  software  required 
to  reformat  input  files,  data  analysis  was  carried  out  using  software  packages  described  in  the  Mathematical  and 
Statistical  Libraiy  of  the  Armstrong  Laboratoiy.  This  libraiy  is  reviewed  in  Albert,  W.  G.  &  Whitehead,  L.  K., 
MATHEMATICAL  AND  STATISTICAL  SOFTWARE  INDEX:  SRCQND  F.DTTTON  (AFHRL-TP-85-47,’ 
August  1986).  This  index  provides  a  brief  description  of  each  package  and  external  references  when  available' 
Detailed  program  documentation  is  only  available  on  the  Unisys  under  their  “@DA*DA.  ADOC”  retrieval  system. 

(AL/HR  internet  address:  “http://www.brooks.af  rnil/HSC/AL/HR/hr-homs.hmil”) 

The  Unisys  1 100  is  scheduled  for  shut-down  and  replacement  in  October  1997  by  an  IBM  RISC  machine 
at  AL/HR  which  is  already  in  operation,  but  not  yet  accessible  to  outside  users.  It  is  unknown  to  this  contractor  as 
to  which  Unisys-resident  programs  are  being  converted  for  operation  on  the  RISC  platform  and  which  will  be  “lost”. 
Metrica  has  already  produced  the  RISC-based  version  of  ASCII  CODAP. 

This  project  used  the  following  packages  in  this  order:  TRICOR  (Correlation  and  Regression  Package, 
Albert  &  Whitehead,  1986,  p  23),  GEN-HIER-GRP  (Generate  HIER-GRP  input  data,  Albert  &  Whitehead,  1986’ 
p  10),  HIER-GRP  (Regression  Equation  Grouping,  Albert*  Whitehead,  1986,  p  11),  GRPREL  (Group  Reliability^ 
Staley  &  Weissmuller,  1981),  and  the  ASCH  CODAP  system  (the  Comprehensive  Occupational  Data  Analysis 

Programs,  Albert  &  Whitehead,  1986,  p  53).  A  description  of  each  package  and  its  applicability  to  this  project  is 
provided  below. 


TRICOR 

The  first  package  used  was  TRICOR  which  stands  for  Triangular  Correlation  &  Regression  system.  This 
system  was  developed  by  the  United  States  Air  Force  Human  Resources  Laboratoiy  (AFHRL)  which  is  now  known 

as  the  HumanResourceDirectorateoftheUSAFAimstrongLaboratoiy.  This  system  can  handle  up  to 400variables 

in  correlation  matrices  and  regression  analysis  problems.  Its  matrix-inversion  (exact  solution)  method  accounts  for 
its  speed  in  processing  hundreds  of  queued  regression  problems  while  screening-out  linear  dependencies  while 
could  lead  to  spuriously  high  results.  In  addition  to  hard-copy  reports,  TRICOR  produces  a  Regression  Master  File 
which  tracks  every  problem  submitted  to  the  system.  This  Regression  Master  File  is  input  to  the  following  program 
which  extracts  all  the  required  statistics  without  error-prone  human  transcription. 

In  this  research  stream,  “rater  policy”  is  defined  in  terms  of  “level  of  comfort”  associated  with  flying  into 
a  given  weather  scenario  and  the  “worth  functions”  the  pilot  gives  to  the  weather  attributes  being  varied  in  this  study 
which  are  ceiling,  visibility,  and  precipitation.  In  statistical  terms,  the  “level  of  comfort”  is  the  criterion  being 
predicted  in  the  regression  equation  and  the  “worth  functions”  are  the  regression  weights  being  computed  for  each 
of  the  predictors  of  ceiling,  visibility,  and  precipitation. 


A1 


In  judging  their  own  personal  comfort  level,  some  people  may  evaluate  the  weather  factors  in  the  proposed 
scenarios  in  different  ways.  Some  people  may  look  at  each  of the  weather  variables  in  turn,  one  at  a  time,  like  ceiling, 
then  visibility,  then  precipitation.  Other  pilots  may  consider  those  factors  in  pairs,  or  all  three  simultaneously.  For 
example,  a  given  pilot  may  view  ceiling  AND  visibility  as  a  single  factor  which  means  if  either  is  bad,  he  or  she  will 
not  fly,  regardless  of  the  other  value.  These  interaction  forms  introduce  a  non-compensatory  notion  into  normal 
linear  forms  which  permit  multiple  “high”  variables  to  overwhelm  a  lone  “low”  variable.  The  regression  model  used 
in  this  project  had  to  be  expanded  to  include  all  “likely  interactions.”  This  means  that  the  regression  model  used 
was  robust  enough  to  capture  not  only  simple  (compensatory),  one-factor  policies,  but  also,  two-,  three-  (non¬ 
compensatory),  or  hybrid  policies.  The  individual  pilot’s  data  is  evaluated  in  terms  of  all  possible  models  and  the 
“best  fit”  becomes  the  policy  ascribed  to  that  pilot. 

Because  the  goal  was  to  determine  pilot  policy  under  each  of  three  terrains,  the  TRICOR  correlations  and 
regressions  were  run  for  each  of  the  three  terrain  scenarios,  each  with  about  three  hundred  cases.  Each  of  the 
TRICOR  Master  Regression  Files  was  submitted  separately  to  the  following  GEN-HIER-GRP,  HBER-GRP,  and 
CODAP  analyses. 


GRPREL 


GRPREL  is  the  standard  interrater  reliability  in  the  CODAP  system.  For  a  given  list  of  items  rated  by  a  set 
of  Subject  Matter  Experts  (SMEs),  this  program  reports  two  measures  of  interrater  agreement  (R  ^  and  R^).  The 
R  value  indicates  the  reliability  of  the  observed  set  of  ratings  —  while  0. 10  is  considered  a  minimum  for  usable 
rafer  agreement  (lower  values  or  negative  values  indicate  the  presence  of  deviant  raters — typically  those  who  are 
using  a  reversed  rating  scale),  a  value  of  0.20  or  greater  is  desired.  The  R  value  is  driven  by  the  number  of  raters 
actually  used.  Although  an  R  of  0.90  is  usually  desired,  it  may  not  be  practical  in  a  particular  study  because  of  a 
small  number  of  raters  (SMEs^lhat  may  be  subdivided  even  further  into  smaller  groups  based  on  policy  differences. 
The  GRPREL  program  also  computes  means  and  standard  deviations  for  each  item  in  the  list.  Item-level  reports 
are  printed  in  three  orders:  original  sequence,  ordered  descending  y-mean  value,  and  ordered  descending  on 
standard  deviation.  GRPREL  computes  each  rater’s  correlation  with  the  full-group  mean  vector  and  uses  a 
probability  evaluation  to  recommend  the  removal  of  deviant  (non-cooperative  or  reversed  scale)  raters.  The 
program  can  automatically  iterate  and  remove  flagged  raters  until  either  a  sufficienct  level  of  agreement  (R^ = 0.90) 
is  reached  or  no  raters  can  be  found  with  a  probability  (of  deviant  rating)  above  0.95. 


GEN-fflER-GRP 

Because  of  the  complexity  and  magnitude  of  the  data  required  by  the  HIER-GRP  program  below,  another 
program,  GEN-HIER-GRP  was  created  to  mechanicaly  reformat  data  from  regression  packages  into  the  form 
required'by  HIER-GRP.  While  the  Albert  et  al  reference  indicates  that  GEN-HIER-GRP  only  accepts  a  TRICOR 
Regression  Master  File  as  input,  personal  communication  with  Janice  Buchhom  of  AL/HR  indicates  the  program 
MAY  have  been  modified  to  accept  matrix  information  as  output  by  SPSS.  Note  that  the  documentation  indicates 
die  software  is  limited  to  200  cases.  Metrica  provided  Ms.  Buchhom  the  source  code  updates  to  increase  this  limit 
to  400  cases  to  accommodate  this  project. 
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HffiR-GRP 


Given  the  regression  analysis  conducted  above  (determination  of  regression  weights  and  “constant”), 
performs  a  hierarchical  clustering  which  begins  with  individuals  (i.e.,  their  policy/regression  equation) 
and  iteratively  forms  larger  and  larger  policy  groups  until  a  single  group  remains.  At  each  stage  of  the  process,  the 
two  most  similar  people  (or  groups)  are  combined  into  a  single,  group  policy  equation.  Two  people  are  considered 
most  similar”  if  the  mathematics  shows  that  (atthat  stage  of  the  process)  combining  those  two  people  loses  the  least 
amount  of  predictive  efficiency,  i.e.,  losses  are  minimized  compared  to  all  other  alternative  pairings.  By  studying 
diagrams  of  the  clustering  stages,  it  can  clearly  be  seen  when  fairly  large  groups  are  being  “forced”  together  by  the 
precipitous  drop  in  the  amount  of  variance  accounted  for  (i.e.,  the  value.)  Note  that  the  documentation  indicates 
the  software  is  limited  to  200  cases.  Metrica  provided  the  source  code  updates  to  increase  this  limit  to  400  cases 
to  accommodate  this  project. 

fflER-GRP  was  run  once  for  each  terrain  scenario.  While  HIER-GRP  provided  a  text-output  report 
identifying  significantly  different  branches  within  each  terrain,  the  cluster  solution  data  for  each  terrain  was 
reformatted  for  display  in  CODAP  below. 


CODAP 


CODAP  stands  for  the  Comprehensive  Occupational  Data  Analysis  Programs  and  represents  over  100 
programs  designed  to  input,  process,  organize  and  report  job-related  data.  CODAP  includes  procedures  for 
clustering  jobs  into  structured  job  families,  performing  interrater  reliability  assessments  on  expert  raters,  comparing 
and  reporting  job  descriptions  for  selected  groups,  reporting  within-group  frequency  distributions  for  selected 
variables  as  well  as  statistical  comparisons  between  groups.  Notethatthe  reference,  Albert  &  Whitehead  is  outdated 
and  program  names  have  changed.  For  current  program  names  and  information,  access  the  internet  URL  “http:// 
metricanet.com/groups/codap”. 


For  the  remainder  of  this  project,  the  primary  data  processing  was  moved  to  the  RISC  machine  for 
processing  with  the  RISC  version  of  CODAP.  Only  the  experimental  versions  of  VARSUM  and  CONHGP 

(explained  below)were  run  on  theUnisysafterthispoint.Datawere  loaded  intoaCODAP  data  base  usingAUDITR, 

INPSTD,  and  AUDITD  (one  for  each  terrain).  The  interrater  reliability  program  (GRPREL)  was  run  on  each  of  the 
terrain  samples.  The  reliability  levels  were  very  high  indicating  a  strong  consensus  in  prioritizing  the  27  weather 
scenarios  from  “good”  to  “bad”.  Note  that  while  pilots  agreed  on  how  to  prioritize  the  weather  scenarios,  they  still 
could  have  arrived  at  those  decisions  along  different  lines  and  have  different  personal  comfort  level  about  entering 
those  conditions. 
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To  address  the  components  of  their  personal  policies,  the  cluster  solution  from  HIER-GRP  was  converted 
to  a  CODAP  cluster  solution  file  using  CONHGP  (not  documented  on  the  internet).  For  each  cluster  solution  the 
CODAP  DIAGRM  program  was  run  to  display  a  summary  of  the  clustering.  Analyzing  the  DIAGRM  lead  to  the 
identification  of  four  policy  groups  for  each  of the  three  terrains.  Summary  variables  such  as  “average  comfort  level” 
and  “group  membership”  were  computed  and  added  to  the  data  base  using  VARGEN.  The  people  in  “stages  of 
interest”  on  the  DIAGRM  were  identified  using  the  STGJOB  program  while  demographic  groups  (based  on  FAA 
region)  were  identified  using  the  MEMSEL  program.  Once  identified  by  either  method,  the  interrater  reliability 
pro  gram  (GRPREL)  was  run  to  assess  agreement  as  well  as  to  produce  a  mean  vector  of  ratings  from  the  group.  Each 
mean  vector  was  shown  in  graphic  form  using  PLTTSK  (a  derivative  of  PLTVAL).  These  mean  vectors  were  input 
to  FACMAT  to  produce  an  intercorrelation  matrix  showing  how  close  the  results  of  the  policies  are  in  prioritizing 
the  weather  scenarios.  In  addition,  for  each  group  identified,  demographics  were  reported  and  cross-compared  using 
the  VARSUM  (experimental  version  with  t-tests)  program. 


Availability  Notes 


TRICOR: 

While  the  TRICOR  system  may  be  replaced  by  any  reasonable  correlation  and  regression  package  (1 
criterion,  7  predictors),  reformatting  outputs  for  HIER-GRP  may  be  tedious. 


GEN-HIER-GRP  &  HIER-GRP: 

No  known  substitute  is  available  for  the  (GEN-HDER-GRP  &  HIER-GRP)  programs.  Much  of  the 
underlying  math  and  all  known  references  are  contained  in  Ward,  J.  H.  Jr,  Treat,  B.  R.,  &  Albert,  W.  G.  GENERAL 
APPTJCATIQNS  OF  HTF.RARCHICAL  GROUPING  USING  THE  HIER-GRP  COMPUTER  PROGRAM. 
(AFHRL-TP-84-42). 

CODAP: 

While  Metrica  maintains  ASCII  CODAP  for  the  Air  Force,  all  release  authorizations  must  go  through  the 
Air  Force.  Coordinating  the  release  of  ASCII  CODAP  from  the  Air  Force  should  start  by  contacting  the  USAF 
AETC  Occupational  Measurement  Squadron  at  Randolph  AFB  (OMSq).  OMSq  personnel  are  the  operational  users 
of  CODAP  in  the  Air  Force  and  underwrote  the  conversion  from  the  Unisys  to  the  RISC  platform.  Their  home  page 
is  “http://www.omsq.af.mil/omy_pl.htm”. 

Other  versions  of  the  CODAP  system  have  been  programmed  over  the  years  and  include  versions 
known  as  IBM  7040,  IBM  360/370,  Univac/Speny/Unisys  1 100,  CDC  6600,  CODAP  80,  IBM  MVS,  IBM 
VM,  IBM  RISC  and  atCODAP.  The  only  systems  known  to  be  available  at  the  present  time  include  the 
government-owned  ASCB  CODAP  family  (Unisys  1 100,  IBM  MVS,  and  RISC  CODAP)  and  the  commercial, 
pc-based  product  atCODAP  from  Sensible  Systems,  Inc.,  also  in  San  Antonio,  Texas. 
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APPENDIX  B 


PILOT  INFORMATION  FORM 


The  aeronautical  charts  used  in  this  study  were  identical  to  those  used  in  the  previous  study  (Driskill 
et  al.,  1997);  therefore,  they  have  not  been  reproduced  here. 
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U.S Department  goo  independence  Ave..  S.W. 

of  Transportoticxi  Washington.  D  C.  20591 

Federal  Aviation 
Administration 


Dear  Airman; 

I  need  your  help.  I  am  conducting  a  study  of  how  pilots  like  you  use  weather 
information  in  making  decisions.  As  you  probably  know,  weather  is  a  significant 
factor  in  a  very  large  number  of  aviation  accidents  every  year.  The  results  of  this 
study  will  help  us  develop  new  and  more  effective  training  programs  to  improve 
safety. 

This  study  examines  how  pilots  use  information  on  ceiling,  visibility,  and 
precipitation  in  their  flight  planning.  It  also  examines  how  terrain  influences  the 
pilot's  use  of  those  weather  data.  Briefly,  the  study  asks  you  to  rank  order 
various  combinations  of  these  weather  factors  from  most  to  least  comfortable  and 
to  then  assign  specific  numerical  ratings  to  each  weather  scenario.  This  process 
is  repeated  for  three  routes  over  widely  different  terrain.  When  we  ran  this  study 
with  some  local  pilots  it  took  them  about  an  hour  to  complete  the  exercise. 

This  is  not  an  easy  exercise.  You  will  really  have  to  think  about  just  what 
conditions  you  would  rather  fly  under,  and  what  factors  are  most  important  to  you 
in  making  weather  decisions.  However,  the  more  we  learn  about  this  decision 
making  process,  the  safer  we  can  make  flying  for  everyone,  so  I  hope  you  will 
take  the  time  to  complete  the  materials  and  send  them  back  to  me.  If  you  have 
any  questions  you  can  reach  me  at  (202)  366-6935. 


Sincerely, 


David  R.  Hunter,  Ph.D. 
Program  Scientist 


YOUR  PARTICIPATION  IS  VOLUNTARY.  If  you  do  not  wish  to  participate, 
simply  discard  these  materials.  If  you  decide  to  participate,  your  responses  will 
remain  anonymous.  There  are  no  codes  or  other  identifying  marks  on  the 
materials  you  are  asked  to  complete  and  return  to  us  that  would  identify  you 
personally. 

This  data  collection  is  covered  under  0MB  Approval  Number  2120-0587 
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INSTRUCTIONS 


First,  let  me  make  it  clear  that  we  are  not  asking  whether  or  not  you  would  actually 
make  the  flights  described  in  these  scenarios.  Some  pilots,  for  example,  would  never 
fly  a  single-engine  aircraft  across  Lake  Michigan,  regardless  of  the  weather  conditions. 
For  this  study,  it  does  not  matter  whether  or  not  you  would  make  a  particular  flight. 

What  is  important  is  how  you  think  the  various  scenarios  compare  against  each  other. 

In  other  words,  if  you  were  flying  along  the  routes  shown,  which  weather  combination 
would  you  moi  like  to  fly  in  and  which  would  be  your  last  choice.  (By  the  way,  those 
are  usually  the  easy  combinations  to  pick  out.  The  going  gets  much  tougher  when 
deciding  on  all  the  in-between  combinations.) 

Since  you  may  never  have  flown  over  water  long  distances  or  through  high  mountain 
areas  or  through  some  of  the  weather  conditions,  you  may  have  to  use  your 
imagination  and  think  about  what  flying  those  routes  under  those  conditions  might  be 
like.  For  a  few  of  the  weather  combinations  (such  as  heavy  snow  and  five  miles 
visibility),  you  may  also  have  to  use  your  imagination,  since  such  a  combination  may  be 
somewhat  unlikely. 

Remember,  there  are  no  right  or  wrong  answers.  We  are  not  trying  to  trick  you  into 
admitting  you  would  fly  under  illegal  conditions.  We  just  want  to  know  how  you  think 
these  combinations  of  weather  rate,  and  how  comfortable  you  would  be  flying  through 
them. 


1 .  Look  at  the  Map  for  Route  1 .  It  shows  a  flight  from  the  City  County  Airport  across 
Lake  Michigan  to  the  Delta  County  Airport  in  Escanaba,  Michigan.  Familiarize  yourself 
with  this  route  of  flight  and  read  the  information  about  your  aircraft  and  the  departure 
and  destination  weather  printed  in  the  insert  box  on  the  map. 

2.  Look  at  the  response  sheet  for  Route  1  which  shows  the  Enroute  Weather.  You  will 
see  that  a  weak  cold  front  is  across  your  route  of  flight  and  that  several  combinations  of 
reported  ceilings,  visibility  and  precipitation  are  given.  For  example,  in  the  row  for 
Scenario  101,  the  ceiling  is  reported  to  be  4,000  feet,  the  visibility  is  7  miles,  and 
moderate  rain  has  been  reported. 

3. 1  Your  first  task  is  to  decide  under  which  set  of  conditions  you  would  be  most 
comfortable  flying  this  particular  route.  Look  carefully  over  all  the  combinations  of 
ceiling,  visibility,  and  precipitation  and  find  the  combination  under  which  you  would  be 
most  comfortable  flying.  When  you  have  found  that  combination,  write  the  number  1 
in  the  box  headed  Rank  Order  for  that  combination 
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INSTRUCTIONS  -  Page  2 


3.2  As  you  fill  in  the  boxes  on  the  forms,  please  print  the  numbers  clearly.  If  there  are 
two  boxes  for  a  particular  answer  (like  there  are  for  the  Rank  Order  responses)  and  you 
want  to  enter  a  single  number  (for  instance,  you  want  to  enter  the  number  1  for  your 
most  comfortable  set  of  conditions)  be  sure  to  put  that  number  in  the  box  on  the  right. 

Example;  Your  entry  for  Rank  Order  1  would  be:  _ 


Your  entry  for  Rank  Order  15  would  be: 


Note  that  you  do  not  need  to  fill  in  the  leading  boxes  with  zeros. 

3.3  Now  look  over  the  remaining  combinations  of  ceiling,  visibility  and  precipitation  and 
find  the  combination  under  which  you  would  be  most  comfortable  flying  from  those 
remaining.  Once  you  have  found  it.  write  the  number  2  in  the  box  headed  Rank  Order 
for  that  combination. 

3.4  Then  just  keep  on  narrowing  the  list  of  combinations  down  by  finding  the 
combination  from  among  those  remaining  under  which  you  are  most  comfortable  and 
marking  that  combination  with  the  next  number.  Finally,  you  will  end  up  with  only  one 
combination  left,  and  that  will  be  the  combination  under  which  you  would  be  least 
comfortable  flying.  If  you  have  done  the  numbering  properly,  that  combination  will  get 
numbered  27. 

3.5  At  this  point  you  should  have  gone  through  all  the  combinations  and  rank  ordered 
them  in  terms  of  your  comfort  so  that  the  combination  you  are  most  comfortable  with 
is  numbered  1  and  the  combination  you  are  least  comfortable  with  is  numbered  27. 

All  the  other  combinations  will  have  some  number  between  those  two  values,  and  no 
value  will  appear  more  than  once.  That  is,  you  can't  use  the  same  rank  number  on 
more  than  one  combination. 

3.6  Now,  look  back  at  the  various  weather  combinations  and  decide  which  combination 
is  the  WORST  COMBINATION  under  which  you  would  continue  the  flight.  That  is,  if 
any  of  the  three  factors  (ceiling,  visibility,  or  precipitation)  got  any  worse,  then  you 
would  not  continue  the  flight  and  would  divert.  Once  you  have  decided  which  is  the 
worst  combination  in  which  you  would  still  fly,  write  the  SCENARIO  NUMBER  of  that 
combination  in  the  box  marked  WORST  FLYABLE  SCENARIO.  If  you  would  not  make 
this  flight  under  any  of  the  combinations  we  have  given,  then  enter  "0  0  0"  in  the  box 
marked  WORST  FLYABLE  SCENARIO 
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INSTRUCTIONS  -  Page  3 


4.1  Your  second  task  is  to  assign  a  level  of  comfort  to  each  individual  scenario 
using  a  scale  from  1  to  100,  where: 

1  =  Most  Comfortable  about  completing  the  flight  VFR  under  these  enroute 
weather  and  terrain  conditions. 

100  =  Least  Comfortable  about  completing  the  flight  VFR  under  the  enroute 
weather  and  terrain  conditions. 

4.2  Use  any  numbers  in  the  range  from  1  to  100  to  indicate  your  level  of  comfort,  but 
you  May  Not  use  the  same  number  more  than  once.  If,  for  instance,  you  feel  that  two 
choices  are  both  "least  comfortable"  then  rate  one  as  "99"  and  the  other  as  "100". 

5.1  After  you  have  rank  ordered  all  the  weather  scenarios  for  Route  1  and  have 
assigned  a  comfort  rating  to  each  scenario  for  that  route,  do  the  same  thing  for  Route 
2  and  Route  3. 

5.2  Beginning  with  Route  2,  review  the  map  and  read  the  information  on  the  map. 

Then  rank  order  each  of  the  combinations  of  weather  conditions.  After  you  have 
completed  the  rank  ordering,  decide  on  the  worst  flyable  scenario,  then  assign  a 
comfort  rating  to  each  combination,  again  using  the  1  to  100  scale. 

5.3  After  you  have  completed  Route  2,  go  on  to  Route  3  and  repeat  the  rank  ordering 
and  comfort  rating  process  for  that  route. 


6.1  To  help  in  the  analysis  of  the  weather  ratings,  we  need  some  information  about  you 
(for  example,  your  age),  your  aviation  ratings  and  experiences,  and  what  area  of  the 
country  you  live  in  (we  get  that  from  the  Zip  code),  so  please  complete  the  sheet 
labeled  Pilot  Background  Data.  Do  not  put  your  name  or  other  identifying  information 
on  the  sheet. 

6.2  If  you  have  any  comments  or  suggestions,  please  feel  free  to  write  them  on  a 
separate  sheet  of  paper  and  send  it  back  to  us. 

7.1  Put  the  three  Weather  Information  Study  sheets  on  which  you  have  written  your 
rank  order  and  comfort  ratings  and  the  Pilot  Background  Data  Sheet  in  the  self- 
addressed  business  reply  envelope  and  drop  it  in  the  mail.  No  postage  is  required. 

That's  all. 


Thank  you  for  taking  the  time  to  help  me  on  this  project.  If  you  would  like 
information  on  the  results  of  this  study,  Just  send  me  a  note  with  your  name  and 
address  (don't  put  them  on  your  answer  sheets)  and  I  will  send  you  a  copy  of  the 
report. 
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Weather  Information  Study 
Route  1  -  City  County  to  Delta  Count 
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H  Weather  Information  Study  ■ 

^ _ Route  2  -  Hereford  Airport  to  Dalhart  Airport _ _ 

Scenario  Ceiling  Visibility  Precipitation  Rank  Comfort  Scenario  Ceiling  Visibility  Precipitation  Rank  Comfort 

Number  (feet)  (miles)  Order  Level  Number  (feet)  (miles)  Order  Level 
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APPENDIX  C 


Regression  Models 

Predicting  Policy-Related  Metrics  from  Demographics 
Regression  Model  1 : 

Predicting  (COOl  1)  Average  Comfort  Level 

r  =  0.39471 
RSq  =  0.15580 

Note:  “ZOC”  stands  for  Zero-Order  Correlation  with  ... 


Variable  ID  (Predictor) 


I  ZOC  with  Comfort  Level 

I  I  ZOC  with  Person’s  RSq  -  Degree  Expert  Info  was  used) 
1  I  I  Unique  Variance  Accounted  for  by  this  Variable 

I  I  I  I  Variable  Description  (Predictor) 

I  I  I  I  I 

VAR  w/COOl  1  w/RSQl  Unique  | 


V0025  .1905 
V0009  -.1848 
C0023-.1409 
V0026  .1250 
C0015  .1235 
V0004  .1144 
V0014  .1141 
V0028  .0949 
V0017-.0884 
V0005  .0701 
C0018  .0612 
COO 16  .0609 
C0020  -.0557 
V0027  .0524 
C0013  .0487 
V0007  -.0399 
C0021  .0210 
V0003  .0202 
V0008  .0174 
V0020  -.0163 


.020353  EXPERIENCE  -  VISIBILITY  4  MILES 
.017303  LOWEST  CEILING  FOR  DAY  VFR  TRIP 
.023269  ROUTE  3  (MOUNTAIN) 

.001216  EXPERIENCE  -  VISIBILITY  8  MILES 

.016317  ALASKA  REGION 

.017114  MULTI-ENGINE  RATING  (1=YES,  0=NO) 

.003797  PCT  TIME  OBTAIN  WEATHER  INFO  -  FAA/FSS 

.004001  EXPERIENCE  -  CEILING  1800  FEET 

.004344  PCT  TIME  OBTAIN  WEATHER  INFO  -  OTHER 

.002925  EMPLOYED  AS  A  PILOT  (1=YES,  0=NO) 

.007541  WESTERN  PACIFIC  REGION 

.001354  SOUTHERN  REGION 

.001034  EASTERN  REGION 

.002153  EXPERIENCE  -  CEILING  800  FEET 

.000873  HIGHEST  CERTIFICATE  -  COMMERCIAL 

.001431  OWN  AIRCRAFT  (1=YES,  0=NO) 

.002777  ROUTE  1  (WATER) 

.001264  INSTRUMENT  RATING  (1=YES,  0=NO) 

.000893  NUMBER  OF  FAA  SAFETY  SEMINARS 
.003723  EXPERIENCE  -  ROUTE  3 


Cl 


coo 14  -.0070 


.0051 19  HIGHEST  CERTIFICATE  -  AIRLINE  TRANSPORT 


Regression  Model  2: 

Predicting  (RSQl)  Person’s  RSq  -  Degree  Expert  Info  was  used. 
(Model  without  using  Comfort  Level  as  a  predictor.) 

r  =  0.38716 
RSq  =  0.14989 

Note:  “ZOC”  stands  for  Zero-Order  Correlation  with  ... 


Variable  ID  (Predictor) 

I  ZOC  with  Comfort  Level 

I  I  ZOC  with  Person’s  RSq  -  Degree  Expert  Info  was  used) 

I  I  I  Unique  Variance  Accounted  for  by  this  Variable 

I  I  I  I  Variable  Description  (Predictor) 

I  I  I  I  I 

VAR  w/COOl  1  w/RSQl  Unique  | 

I  I  I  I  I 

C0021  -.2612  .069710  ROUTE  1  (WATER) 

VOOlO  -.0994  .001646  AGE 

VOO 14  -.0988  .006845  PCT  TIME  OBTAIN  WEATHER  INFO  -  FAA/FSS 

V0019  .0930  .007995  EXPERIENCE  -  ROUTE  2 

V0026  .0790  .005269  EXPERIENCE -VISIBILITY  8  MILES 

C0020  .0759  .000958  EASTERN  REGION 

COO  16  -.0686  .002257  SOUTHERN  REGION 

C0022  .0659  .005700  ROUTE  2  (FLAT) 

COO  15  .0624  .003744  ALASKA  REGION 

C0014  .0618  .009200  HIGHEST  CERTIFICATE -AIRLINE  TRANSPORT 

V0022  -.0612  .006049  EXPERIENCE  -  MODERATE  RAIN 

V0024  -.0580  .001329  EXPERIENCE  -  VISIBILITY  1  MILE 

C0013  -.0579  .001243  HIGHEST  CERTIFICATE -COMMERCIAL 

VOO  13  -.0525  .003875  TOTAL  HOURS  LOGGED 

C0017  -.0445  .001455  NEW  ENGLAND  REGION 

V0006  -.0429  .004380  CFI  OR  CFII  CERTIFICATE  (1=YES,  0=NO) 

V0023  -.0416  .000503  EXPERIENCE -HEAVY SNOW 

VOO  12  .0390  .001357  HOURS  LOGGED  IN  LAST  90  DAYS 

V0028  .0319  .000603  EXPERIENCE  -  CEILING  1800  FEET 

VOO  1 6  .0158  .002992  PCT  TIME  OBTAIN  WEATHER  INFO  -  TV/RADIO 

V0021  -.0083  .001260  EXPERIENCE  -  LIGHT  RAIN 


C2 


Regression  Model  3: 

Predicting  (RSQl)  Person  s  RSq  -  Degree  Expert  Info  was  used. 
(Model  using  Comfort  Level  as  a  predictor.) 


r  =  0.41940 
RSq  =  0.17590 


Note:  "ZOC”  stands  for  Zero-Order  Correlation  with  ... 


Variable  ID  (Predictor) 

[  ZOC  with  Comfort  Level 

I  I  ZOC  with  Person’s  RSq  -  Degree  Expert  Info  was  used) 

I  I  I  Unique  Variance  Accounted  for  by  this  Variable 

I  I  I  I  Variable  Description  (Predictor) 

I  I  I  I  I 

VAR  w/COOll  w/RSQl  Unique  | 

I  I  I  I  I 

C0021  .0205  -.2612  .060171  ROUTE  1  (WATER) 

cool  1  1.0000  -.1640  .026108  AVERAGE  COMFORT  RATING 

VOOlO -.0103 -.0994  .001723  AGE 

V0014  .1038 -.0988  .005004  PCT  TIME  OBTAIN  WEATHER  INFO  -  FAA/FSS 

V0019  .0480  .0930  .008659  EXPERIENCE  -  ROUTE  2 

V0026  .1742  .0790  .010557  EXPERIENCE -VISIBILITY  8  MILES 

C0020  -.0468  .0759  .004221  EASTERN  REGION 

C0022  .1189  .0659  .002370  ROUTE  2  (FLAT) 

C0015  .1124  .0624  .010397  ALASKA  REGION 

C0014-.0175  .0618  .005369  HIGHEST  CERTIFICATE  -  AIRLINE  TRANSPORT 

V0022  .0899 -.0612  .005670  EXPERIENCE  -  MODERATE  RAIN 

C0013  .0635 -.0579  .000598  HIGHEST  CERTIFICATE  -  COMMERCIAL 

V0013  .0378 -.0525  .003143  TOTAL  HOURS  LOGGED 

V0006  .0481  -.0429  .005361  CFI  ORCFII  CERTIFICATE  (1=YES,  0=NO) 

V0023  .0610 -.0416  .000997  EXPERIENCE  -  HEAVY  SNOW 

V0012  .0070  .0390  .001398  HOURS  LOGGED  IN  LAST  90  DAYS 

V0027  .0474  -.0341  .001608  EXPERIENCE  -  CEILING  800  FEET 

V0009  -.1898  -.0273  .002502  LOWEST  CEILING  FOR  DAY  VFR  TRIP 

V0016  -.0527  .0158  .002957  PCT  TIME  OBTAIN  WEATHER  INFO  -  TV/RADIO 

V0021  .0766 -.0083  .001507  EXPERIENCE -LIGHT  RAIN 

V0004  .0974  .0058  .001805  MULTI-ENGINE  RATING  (1=YES,  0=NO) 

C0018  .0722  .0045  .003173  WESTERN  PACIFIC  REGION 
C0019  -.0828  -.0009  .001 1 1 1  NORTHWEST  MOUNTAINS  REGION 
V0020  -.0149  -.0003  .000836  EXPERIENCE  -  ROUTE  3 
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